Sep 11 04:38:06.670227 ip-10-0-139-137 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 04:38:06.670239 ip-10-0-139-137 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 04:38:06.670247 ip-10-0-139-137 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 04:38:06.670537 ip-10-0-139-137 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 04:38:16.255270 ip-10-0-139-137 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 11 04:38:16.255341 ip-10-0-139-137 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d777b2a6ff4f4e8fadd259e65fbc60f6 --
Sep 11 04:40:04.238186 ip-10-0-139-137 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 04:40:04.634730 ip-10-0-139-137 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:40:04.634730 ip-10-0-139-137 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 04:40:04.634730 ip-10-0-139-137 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:40:04.634730 ip-10-0-139-137 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 04:40:04.634730 ip-10-0-139-137 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:40:04.636507 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.636430    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 04:40:04.639249 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639234    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.639249 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639250    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639254    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639257    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639259    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639262    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639265    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639268    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639271    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639275    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639279    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639282    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639285    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639288    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639291    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639293    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639296    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639298    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639301    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639304    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639307    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.639307 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639310    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639313    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639316    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639318    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639321    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639324    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639326    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639329    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639332    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639334    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639337    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639339    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639342    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639344    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639347    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639350    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639352    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639354    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639357    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.639798 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639360    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639364    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639367    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639370    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639373    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639375    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639378    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639380    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639383    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639385    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639388    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639390    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639393    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639395    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639397    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639401    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639404    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639406    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639409    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.640253 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639411    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639414    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639416    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639419    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639421    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639424    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639427    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639431    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639434    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639436    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639438    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639441    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639444    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639446    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639449    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639451    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639454    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639456    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639459    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.640753 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639461    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639463    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639466    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639469    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639471    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639474    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639477    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639479    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639482    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639486    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639489    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639492    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639494    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639497    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639500    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639502    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639505    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639507    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639509    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639512    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.641204 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.639514    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640209    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640217    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640220    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640223    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640225    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640228    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640231    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640234    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640237    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640240    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640242    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640244    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640247    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640250    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640252    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640255    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640257    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640260    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640263    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.641706 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640266    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640269    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640272    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640275    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640277    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640280    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640282    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640285    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640287    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640290    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640293    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640297    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640299    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640302    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640304    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640306    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640309    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640311    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640313    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.642188 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640317    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640319    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640322    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640324    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640326    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640328    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640332    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640336    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640339    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640342    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640344    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640347    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640349    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640352    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640356    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640358    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640361    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640364    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640367    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.642656 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640369    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640373    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640375    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640378    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640380    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640382    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640385    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640387    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640390    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640393    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640396    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640399    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640402    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640404    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640407    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640410    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640413    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640416    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640418    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640421    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.643117 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640423    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640425    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640428    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640431    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640434    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640436    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640438    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640442    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640444    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640447    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640449    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640452    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640454    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640456    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640459    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640461    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640464    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640466    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640468    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640471    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.643590 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640473    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640476    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640833    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640839    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640841    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640844    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640847    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640850    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640852    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640855    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640857    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640860    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640863    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640865    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640868    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640870    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640872    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640875    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640877    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640881    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.644094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640884    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640887    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640889    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640892    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640894    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640897    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640899    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640902    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640904    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640906    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640909    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640911    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640914    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640916    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640918    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640921    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640923    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640925    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640930    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.644578 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640934    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640937    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640940    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640942    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640945    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640947    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640950    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640952    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640954    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640957    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640959    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640961    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640965    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640967    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640970    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640972    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640975    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640977    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640980    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640982    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.645050 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640985    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640987    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640990    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640992    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640995    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640997    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.640999    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641002    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641004    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641007    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641009    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641012    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641014    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641017    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641020    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641022    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641024    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641027    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641030    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641032    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.645568 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641035    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641037    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641040    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641042    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641045    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641047    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641050    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641052    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641055    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641057    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641062    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641065    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641068    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641070    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641073    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641076    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641078    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641080    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641083    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.646080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641085    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641089    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641092    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641162    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641196    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641202    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641207    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641211    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641215    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641219    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641223    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641226    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641229    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641232    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641235    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641238    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641241    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641244    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641247    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641250    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641252    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641256    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641259    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 04:40:04.646540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641262    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641265    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641268    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641272    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641274    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641278    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641281    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641283    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641286    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641289    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641294    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641297    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641301    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641303    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641306    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641309    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641311    2581 flags.go:64] FLAG: --enable-server="true"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641314    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641318    2581 flags.go:64] FLAG: --event-burst="100"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641321    2581 flags.go:64] FLAG: --event-qps="50"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641324    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641327    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641329    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641332    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 04:40:04.647085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641335    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641338    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641342    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641345    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641347    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641350    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641353    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641356    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641359    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641361    2581 flags.go:64] FLAG: --feature-gates=""
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641365    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641367    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641370    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641373    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641376    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641379    2581 flags.go:64] FLAG: --help="false"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641381    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641384    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641388    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641391    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641394    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641397    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641400    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 04:40:04.647678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641402    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641405    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641408    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641411    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641414    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641416    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641419    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641422    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641425    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641427    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641430    2581 flags.go:64] FLAG: --lock-file=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641435    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641438    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641440    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641445    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641448    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641451    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641453    2581 flags.go:64] FLAG: --logging-format="text"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641456    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641459    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641462    2581 flags.go:64] FLAG: --manifest-url=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641465    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641469    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641471    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641475    2581 flags.go:64] FLAG: --max-pods="110"
Sep 11 04:40:04.648295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641478    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641480    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641485    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641487    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641490    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641493    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641496    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641503    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641505    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641508    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641511    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641514    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641519    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641521    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641524    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641527    2581 flags.go:64] FLAG: --port="10250"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641530    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641532    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02ca2223ad2fbfa00"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641536    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641540    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641543    2581 flags.go:64] FLAG: --register-node="true"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641546    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641549    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641552    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 11 04:40:04.648959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641554    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641557    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641560    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641563    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641566    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641569    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641572    2581 flags.go:64] FLAG: --runonce="false"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641574    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641577    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641580    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641584    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641587    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641590    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641593    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641596    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641598    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641601    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641604    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641607    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641610    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641629    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641632    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641638    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641641    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641644    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 04:40:04.649541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641647    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641650    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641654    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641657    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641660    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641662    2581 flags.go:64] FLAG: --v="2"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641666    2581 flags.go:64] FLAG: --version="false"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641670    2581 flags.go:64] FLAG: --vmodule=""
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641673    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.641676    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641787    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641791    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641795    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641798    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641803    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641807    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641809    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641814    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641816    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641819    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641821    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.650261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641824    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641826    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641829    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641831    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641834    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641836    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641839    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641842    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641845    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641847    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641850    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641852    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641855    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641858    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641861    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641864    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641867    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641869    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641871    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641874    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.650807 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641877    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641879    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641882    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641884    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641886    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641889    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641891    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641895    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641899    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641902    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641905    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641907    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641909    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641912    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641915    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641917    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641920    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641922    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641925    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641927    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.651286 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641932    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641935    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641937    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641940    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641942    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641946    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641949    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641951    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641953    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641956    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641959    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641961    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641964    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641966    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641969    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641971    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641973    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641976    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641980    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.651801 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641983    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641988    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641991    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641994    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641996    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.641999    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642002    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642005    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642009    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642013    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642016    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642019    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642022    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642024    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642028    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642030    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642033    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642035    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642039    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642041    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.652265 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642044    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642046    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642049    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642051    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642054    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642056    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642059    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642061    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.642064    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.642924    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.649896    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.649910    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.651310    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.651320    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651373    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.652800 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651378    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651381    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651384    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651387    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651390    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651393    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651395    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651398    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651400    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651402    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651405    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651407    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651410    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651412    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651415    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651417    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651420    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651422    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651425    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651427    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.653352 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651430    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651432    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651435    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651437    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651439    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651442    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651445    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651448    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651452    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651454    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651457    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651459    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651462    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651464    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651467    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651469    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651472    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651474    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651476    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651479    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.654107 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651482    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651486    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651489    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651491    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651494    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651496    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651499    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651501    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651504    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651506    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651510    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651515    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651518    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651520    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651523    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651526    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651528    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651531    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651535    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.654589 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651538    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651541    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651544    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651547    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651550    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651552    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651555    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651557    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651560    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651563    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651567    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651569    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651572    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651575    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651578    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651581    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651583    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651585    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651588    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651591    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.655137 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651594    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651597    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651599    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651602    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651604    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651607    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651609    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651611    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651631    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651635    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651646    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651650    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651653    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651656    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651658    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651662    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651665    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651668    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.655652 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651670    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.651675    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651788    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651793    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651796    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651799    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651802    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651805    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651808    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651810    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651813    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651816    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651818    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651821    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651823    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:40:04.656088 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651826    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651828    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651830    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651833    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651836    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651838    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651840    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651842    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651845    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651847    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651850    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651852    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651855    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651858    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651861    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651864    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651867    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651870    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651872    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:40:04.656477 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651875    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651877    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651879    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651882    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651884    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651887    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651889    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651892    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651894    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651896    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651899    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651901    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651903    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651906    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651908    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651911    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651913    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651915    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651917    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651920    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:40:04.656955 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651922    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651925    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651927    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651929    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651932    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651936    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651939    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651942    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651945    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651948    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651950    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651953    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651955    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651958    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651960    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651963    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651965    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651967    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651970    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651972    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:40:04.657525 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651975    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651977    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651979    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651982    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651984    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651987    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651989    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651991    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651994    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651996    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.651998    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652001    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652003    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652005    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652008    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652010    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652013    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652015    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652018    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:40:04.658025 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652021    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652024    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652027    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652030    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652033    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652035    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652038    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:04.652040    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.652045    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.652136    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.656259    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.657216    2581 server.go:1019] "Starting client certificate rotation"
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.657323    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 04:40:04.658522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.657375    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 04:40:04.659683 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.659611    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 04:40:04.660658 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.660642    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 04:40:04.676297 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.676279    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 11 04:40:04.678890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.678877    2581 log.go:25] "Validated CRI v1 image API"
Sep 11 04:40:04.680791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.680765    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 04:40:04.683479 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.683454    2581 fs.go:135] Filesystem UUIDs: map[1311d623-268f-45e7-8c50-18a226d547ad:/dev/nvme0n1p4 16d1228a-e69d-4ca5-9b44-9ce8c8cde178:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 11 04:40:04.683545 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.683478    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 04:40:04.684230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.684215    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 04:40:04.689242 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.689144    2581 manager.go:216] Machine: {Timestamp:2026-09-11 04:40:04.687839701 +0000 UTC m=+0.337276681 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114562 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec225bf6b11e8c2ec8cdf4654682f029 SystemUUID:ec225bf6-b11e-8c2e-c8cd-f4654682f029 BootID:d777b2a6-ff4f-4e8f-add2-59e65fbc60f6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:40:6e:c9:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:40:6e:c9:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:23:b6:44:76:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 04:40:04.689242 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.689236    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 04:40:04.689354 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.689305    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 04:40:04.692184 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.692165    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 04:40:04.692312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.692188    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 04:40:04.693503 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.693493    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 04:40:04.693543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.693506    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 04:40:04.693543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.693512    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 04:40:04.694105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.694095    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 04:40:04.694821 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.694812    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 04:40:04.698558 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698547    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 04:40:04.698680 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698671    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 04:40:04.698718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698699    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 04:40:04.698718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698714    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 04:40:04.698768 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698733    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 04:40:04.698768 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698742    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 11 04:40:04.698768 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.698750    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 04:40:04.699073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.699063    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 04:40:04.699102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.699081    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 04:40:04.703256 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.703239    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 04:40:04.705764 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.705751    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 04:40:04.705810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.705780    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 04:40:04.707377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707366    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 04:40:04.707414 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707382    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 04:40:04.707414 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707388    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 04:40:04.707414 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707393    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 04:40:04.707414 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707402    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 04:40:04.707414 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707407    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707420    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707425    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707432    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707437    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707445    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 04:40:04.707547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.707449    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 04:40:04.708157 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.708146    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 04:40:04.708190 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.708159    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 04:40:04.708280 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.708263    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rp45p"
Sep 11 04:40:04.710600 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.710575    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-137.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 04:40:04.711322 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.711301    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 04:40:04.711322 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.711306    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 04:40:04.711881 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.711870    2581 server.go:1268] "Started kubelet"
Sep 11 04:40:04.711941 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.711920    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 04:40:04.712036 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.711982    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 04:40:04.712090 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.712061    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 04:40:04.712722 ip-10-0-139-137 systemd[1]: Started Kubernetes Kubelet.
Sep 11 04:40:04.713196 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.713069    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 04:40:04.714289 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.714277    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 04:40:04.716359 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.716335    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 04:40:04.717487 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.717474    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rp45p"
Sep 11 04:40:04.719734 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.718881    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-137.ec2.internal.18d42a6e2fee5035  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-137.ec2.internal,UID:ip-10-0-139-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-137.ec2.internal,},FirstTimestamp:2026-09-11 04:40:04.711845941 +0000 UTC m=+0.361282921,LastTimestamp:2026-09-11 04:40:04.711845941 +0000 UTC m=+0.361282921,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-137.ec2.internal,}"
Sep 11 04:40:04.720151 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720136    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 04:40:04.720202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720150    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 04:40:04.720285 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.720269    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:04.720757 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720731    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 04:40:04.720837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720776    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 04:40:04.720896 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720734    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 04:40:04.720954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720942    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 04:40:04.720996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.720955    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 04:40:04.721152 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721138    2581 factory.go:55] Registering systemd factory
Sep 11 04:40:04.721225 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721156    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 11 04:40:04.721373 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721360    2581 factory.go:153] Registering CRI-O factory
Sep 11 04:40:04.721423 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721377    2581 factory.go:223] Registration of the crio container factory successfully
Sep 11 04:40:04.721478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721425    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 04:40:04.721478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721443    2581 factory.go:103] Registering Raw factory
Sep 11 04:40:04.721478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.721452    2581 manager.go:1196] Started watching for new ooms in manager
Sep 11 04:40:04.722843 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.722828    2581 manager.go:349] Starting recovery of all containers
Sep 11 04:40:04.723887 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.723861    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 04:40:04.723986 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.723952    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 04:40:04.733548 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.733422    2581 manager.go:354] Recovery completed
Sep 11 04:40:04.737538 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.737524    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.739172 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739157    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.739231 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739184    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.739231 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739194    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.739690 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739675    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.739787 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739674    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 04:40:04.739787 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739704    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 04:40:04.739787 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.739718    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 04:40:04.740307 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.740293    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.740379 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.740323    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.740379 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.740336    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.741821 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.741810    2581 policy_none.go:49] "None policy: Start"
Sep 11 04:40:04.741872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.741825    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 04:40:04.741872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.741835    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 04:40:04.743725 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.743714    2581 policy_none.go:47] "Start"
Sep 11 04:40:04.795173 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795157    2581 manager.go:337] "Starting Device Plugin manager"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.795218    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795231    2581 server.go:87] "Starting device plugin registration server"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795518    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795529    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795636    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795714    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.795722    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.796183    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 04:40:04.809575 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.796230    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:04.826686 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.826668    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 04:40:04.827922 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.827900    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 04:40:04.827988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.827948    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 04:40:04.827988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.827971    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 04:40:04.828072 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.828013    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 04:40:04.830359 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.830343    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:40:04.896303 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.896247    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.896971 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.896957    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.897031 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.896983    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.897031 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.896994    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.897031 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.897012    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.904807 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.904794    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.904848 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.904813    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-137.ec2.internal\": node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:04.920934 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.920912    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:04.928321 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.928301    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"]
Sep 11 04:40:04.928366 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.928357    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.929006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.928992    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.929067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.929022    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.929067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.929034    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.930963 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.930952    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.931084 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931064    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.931126 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931097    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.931572 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931558    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.931649 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931587    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.931649 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931601    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.931649 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931563    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.931742 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931655    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.931742 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.931667    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.933643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.933611    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.933705 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.933652    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:40:04.934255 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.934239    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:40:04.934340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.934268    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:40:04.934340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:04.934282    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:40:04.957165 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.957144    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-137.ec2.internal\" not found" node="ip-10-0-139-137.ec2.internal"
Sep 11 04:40:04.960383 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:04.960369    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-137.ec2.internal\" not found" node="ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.021152 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.021133    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.022280 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.022265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.022330 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.022289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.022330 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.022305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e4442bfe415e79ad545fffb4094f327-config\") pod \"kube-apiserver-proxy-ip-10-0-139-137.ec2.internal\" (UID: \"4e4442bfe415e79ad545fffb4094f327\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.121272 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.121246    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.122415 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.122472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.122472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e4442bfe415e79ad545fffb4094f327-config\") pod \"kube-apiserver-proxy-ip-10-0-139-137.ec2.internal\" (UID: \"4e4442bfe415e79ad545fffb4094f327\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.122472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e4442bfe415e79ad545fffb4094f327-config\") pod \"kube-apiserver-proxy-ip-10-0-139-137.ec2.internal\" (UID: \"4e4442bfe415e79ad545fffb4094f327\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.122575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.122575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.122502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f0cab04a8f401f88ebf3b4b63e59fc8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal\" (UID: \"0f0cab04a8f401f88ebf3b4b63e59fc8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.222266 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.221889    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.260159 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.260137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.262790 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.262772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.322027 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.322002    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.422343 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.422318    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.522814 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.522756    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.623160 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.623139    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.657390 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.657372    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 04:40:05.657791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.657515    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 04:40:05.723480 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:05.723462    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-137.ec2.internal\" not found"
Sep 11 04:40:05.723556 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.723479    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 04:35:06 +0000 UTC" deadline="2028-04-16 21:45:38.693543326 +0000 UTC"
Sep 11 04:40:05.723556 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.723502    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14009h5m32.970043732s"
Sep 11 04:40:05.741919 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.741901    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:40:05.820720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.820670    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.830593 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.830577    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 04:40:05.832423 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.832406    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal"
Sep 11 04:40:05.837841 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.837828    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 04:40:05.897199 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:05.897177    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4442bfe415e79ad545fffb4094f327.slice/crio-9abd4b6a9cc6684ed6301286b0bab66fe8e0c1c75b2741aec0446eb4893342ab WatchSource:0}: Error finding container 9abd4b6a9cc6684ed6301286b0bab66fe8e0c1c75b2741aec0446eb4893342ab: Status 404 returned error can't find the container with id 9abd4b6a9cc6684ed6301286b0bab66fe8e0c1c75b2741aec0446eb4893342ab
Sep 11 04:40:05.898202 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:05.898180    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0cab04a8f401f88ebf3b4b63e59fc8.slice/crio-b00a880d44ef17ea683a02a8d0e225431795ca5e03ed8489ca1987b421365ef8 WatchSource:0}: Error finding container b00a880d44ef17ea683a02a8d0e225431795ca5e03ed8489ca1987b421365ef8: Status 404 returned error can't find the container with id b00a880d44ef17ea683a02a8d0e225431795ca5e03ed8489ca1987b421365ef8
Sep 11 04:40:05.902147 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.902131    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:40:05.993545 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:05.993526    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:40:06.016776 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:06.016761    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:40:08.057711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.057683    2581 apiserver.go:52] "Watching apiserver"
Sep 11 04:40:08.067392 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.067364    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 04:40:08.067812 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.067786    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-hvt4h","openshift-multus/multus-nz8zm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8","openshift-dns/node-resolver-md27b","openshift-multus/network-metrics-daemon-2hqzh","openshift-network-diagnostics/network-check-target-psmdc","openshift-network-operator/iptables-alerter-8zvks","openshift-ovn-kubernetes/ovnkube-node-622zx","kube-system/global-pull-secret-syncer-72h62","kube-system/konnectivity-agent-n5vzf","kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal","openshift-cluster-node-tuning-operator/tuned-nhglp","openshift-image-registry/node-ca-2wtlv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal"]
Sep 11 04:40:08.070396 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.070378    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.073349 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.073329    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.073465 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.073365    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.073646 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.073626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.073745 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.073651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l95dm\""
Sep 11 04:40:08.073865 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.073849    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 04:40:08.076927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.075684    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.076927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.076050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9mqhq\""
Sep 11 04:40:08.076927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.076221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.076927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.076400    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 04:40:08.076927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.076463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.078152 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.078134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.079052 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.079032    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 04:40:08.079302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.079285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gc6gm\""
Sep 11 04:40:08.079495 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.079477    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.079752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.079732    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.079931 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.079912    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 04:40:08.080476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.080459    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.080656 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.080623    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.080800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.080783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qlqh8\""
Sep 11 04:40:08.084136 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.084121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.084208 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.084188    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:08.086449 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.086421    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:08.086528 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.086482    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:08.087542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q5h\" (UniqueName: \"kubernetes.io/projected/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kube-api-access-r8q5h\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-serviceca\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.087666 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087704 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087736 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.087736 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e669f686-df35-48bc-96dc-e7827a511c2f-iptables-alerter-script\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.087793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcs6f\" (UniqueName: \"kubernetes.io/projected/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-kube-api-access-dcs6f\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.087793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd6ab5b1-010f-470d-aed1-b35adf58e938-hosts-file\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.087852 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6ab5b1-010f-470d-aed1-b35adf58e938-tmp-dir\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.087852 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whp77\" (UniqueName: \"kubernetes.io/projected/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-kube-api-access-whp77\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.087852 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.087947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7s8p\" (UniqueName: \"kubernetes.io/projected/dd6ab5b1-010f-470d-aed1-b35adf58e938-kube-api-access-c7s8p\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.088058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e669f686-df35-48bc-96dc-e7827a511c2f-host-slash\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.088058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72zwk\" (UniqueName: \"kubernetes.io/projected/e669f686-df35-48bc-96dc-e7827a511c2f-kube-api-access-72zwk\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.088058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.087994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-host\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.088058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.088022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.089916 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.089852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.090031 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.089922    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:08.092603 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.091859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.092603 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.092453    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 04:40:08.093788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.093771    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 04:40:08.094085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094059    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 04:40:08.094085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094073    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.094239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094104    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 04:40:08.094239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094059    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.094460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094434    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pbb6b\""
Sep 11 04:40:08.094460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.094449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.096373 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.096253    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 04:40:08.096373 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.096315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mk4mp\""
Sep 11 04:40:08.097725 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.097398    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.099486 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.099468    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 04:40:08.099741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.099726    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mcmpx\""
Sep 11 04:40:08.099897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.099883    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 04:40:08.100410 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.100395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.102808 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.102790    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 04:40:08.102885 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.102811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wtcd4\""
Sep 11 04:40:08.102968 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.102948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.108054 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.103245    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.108146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.103283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.108201 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.103487    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 04:40:08.108258 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.103535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 04:40:08.108479 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.103843    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 04:40:08.110479 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.110456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m7hp2\""
Sep 11 04:40:08.110661 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.110646    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 04:40:08.110867 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.110849    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gjcb6"
Sep 11 04:40:08.110951 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.110940    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:40:08.119140 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.119122    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gjcb6"
Sep 11 04:40:08.179694 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.179665    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 04:40:08.188286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd6ab5b1-010f-470d-aed1-b35adf58e938-hosts-file\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.188286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6ab5b1-010f-470d-aed1-b35adf58e938-tmp-dir\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.188286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-k8s-cni-cncf-io\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-kubelet\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e34a4577-5ec7-4373-8b1d-c6b212d224f9-agent-certs\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd6ab5b1-010f-470d-aed1-b35adf58e938-hosts-file\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovn-node-metrics-cert\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-kubernetes\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-host\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-host\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-system-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-sys\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-bin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-systemd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd6ab5b1-010f-470d-aed1-b35adf58e938-tmp-dir\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-netd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.188854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-conf-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e669f686-df35-48bc-96dc-e7827a511c2f-host-slash\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72zwk\" (UniqueName: \"kubernetes.io/projected/e669f686-df35-48bc-96dc-e7827a511c2f-kube-api-access-72zwk\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-systemd-units\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-netns\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.188981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e669f686-df35-48bc-96dc-e7827a511c2f-host-slash\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k2qs\" (UniqueName: \"kubernetes.io/projected/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-kube-api-access-8k2qs\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysconfig\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmbch\" (UniqueName: \"kubernetes.io/projected/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-kube-api-access-pmbch\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brc2s\" (UniqueName: \"kubernetes.io/projected/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-kube-api-access-brc2s\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-multus-certs\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-bin\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-tmp\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cnibin\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.189798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e669f686-df35-48bc-96dc-e7827a511c2f-iptables-alerter-script\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcs6f\" (UniqueName: \"kubernetes.io/projected/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-kube-api-access-dcs6f\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-kubelet-config\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-config\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.189387    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-systemd\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-var-lib-kubelet\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.189455    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:08.689434604 +0000 UTC m=+2.980909042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-kubelet\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-slash\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-var-lib-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-log-socket\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whp77\" (UniqueName: \"kubernetes.io/projected/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-kube-api-access-whp77\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-dbus\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.190476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-multus\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-etc-kubernetes\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e669f686-df35-48bc-96dc-e7827a511c2f-iptables-alerter-script\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e34a4577-5ec7-4373-8b1d-c6b212d224f9-konnectivity-ca\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.189995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-run\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-serviceca\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-socket-dir-parent\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-hostroot\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-etc-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-env-overrides\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-conf\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-node-log\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-host\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.191236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-tuned\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgbtd\" (UniqueName: \"kubernetes.io/projected/ab4490c5-39e9-414e-9a46-944e6350b910-kube-api-access-zgbtd\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-modprobe-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7s8p\" (UniqueName: \"kubernetes.io/projected/dd6ab5b1-010f-470d-aed1-b35adf58e938-kube-api-access-c7s8p\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-multus-daemon-config\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-ovn\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-script-lib\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-serviceca\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q5h\" (UniqueName: \"kubernetes.io/projected/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kube-api-access-r8q5h\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-cnibin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd524708-3d81-4da8-9e10-e8e4874a9a4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.192102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-cni-binary-copy\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.193102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-netns\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.193102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-lib-modules\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.193102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.190870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal" event={"ID":"0f0cab04a8f401f88ebf3b4b63e59fc8","Type":"ContainerStarted","Data":"b00a880d44ef17ea683a02a8d0e225431795ca5e03ed8489ca1987b421365ef8"}
Sep 11 04:40:08.193102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.192018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal" event={"ID":"4e4442bfe415e79ad545fffb4094f327","Type":"ContainerStarted","Data":"9abd4b6a9cc6684ed6301286b0bab66fe8e0c1c75b2741aec0446eb4893342ab"}
Sep 11 04:40:08.197832 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.197805    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 04:40:08.204638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.204569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcs6f\" (UniqueName: \"kubernetes.io/projected/ddecf3f4-042d-4f9f-b94e-c9deff6d68b0-kube-api-access-dcs6f\") pod \"node-ca-2wtlv\" (UID: \"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0\") " pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.204638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.204568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q5h\" (UniqueName: \"kubernetes.io/projected/bd524708-3d81-4da8-9e10-e8e4874a9a4e-kube-api-access-r8q5h\") pod \"aws-ebs-csi-driver-node-xxnv8\" (UID: \"bd524708-3d81-4da8-9e10-e8e4874a9a4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.205001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.204972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7s8p\" (UniqueName: \"kubernetes.io/projected/dd6ab5b1-010f-470d-aed1-b35adf58e938-kube-api-access-c7s8p\") pod \"node-resolver-md27b\" (UID: \"dd6ab5b1-010f-470d-aed1-b35adf58e938\") " pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.205501 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.205469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whp77\" (UniqueName: \"kubernetes.io/projected/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-kube-api-access-whp77\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.206864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.206844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72zwk\" (UniqueName: \"kubernetes.io/projected/e669f686-df35-48bc-96dc-e7827a511c2f-kube-api-access-72zwk\") pod \"iptables-alerter-8zvks\" (UID: \"e669f686-df35-48bc-96dc-e7827a511c2f\") " pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.291055 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.290957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-system-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.291055 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.290986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-sys\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.291055 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-bin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.291055 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-systemd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291055 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-netd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291079    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-conf-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-systemd-units\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-netns\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k2qs\" (UniqueName: \"kubernetes.io/projected/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-kube-api-access-8k2qs\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysconfig\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmbch\" (UniqueName: \"kubernetes.io/projected/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-kube-api-access-pmbch\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brc2s\" (UniqueName: \"kubernetes.io/projected/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-kube-api-access-brc2s\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-multus-certs\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-bin\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-tmp\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cnibin\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.291370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-kubelet-config\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-config\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-systemd\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-var-lib-kubelet\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-kubelet\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-slash\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-var-lib-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-log-socket\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-dbus\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-multus\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-etc-kubernetes\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e34a4577-5ec7-4373-8b1d-c6b212d224f9-konnectivity-ca\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-run\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-socket-dir-parent\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292035 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-hostroot\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-etc-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-env-overrides\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-conf\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-node-log\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-host\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-tuned\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgbtd\" (UniqueName: \"kubernetes.io/projected/ab4490c5-39e9-414e-9a46-944e6350b910-kube-api-access-zgbtd\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-modprobe-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-multus-daemon-config\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-ovn\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-script-lib\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.291986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-cnibin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.292625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-cni-binary-copy\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-netns\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-lib-modules\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-k8s-cni-cncf-io\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-kubelet\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e34a4577-5ec7-4373-8b1d-c6b212d224f9-agent-certs\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovn-node-metrics-cert\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-kubernetes\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-kubernetes\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-system-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-sys\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-bin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-systemd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-netd\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292520    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-cni-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.292749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.293376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-conf-dir\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-run\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-hostroot\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-etc-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.293828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-netns\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysconfig\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-conf\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.294203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-node-log\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-host\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-env-overrides\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-multus-certs\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-cni-bin\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-kubelet\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-slash\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-var-lib-openvswitch\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-etc-kubernetes\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-log-socket\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-dbus\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-modprobe-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-cni-multus\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-k8s-cni-cncf-io\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.294936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.294926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-cnibin\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295110    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e34a4577-5ec7-4373-8b1d-c6b212d224f9-konnectivity-ca\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-multus-socket-dir-parent\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-systemd-units\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-var-lib-kubelet\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-multus-daemon-config\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-run-ovn\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.295437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295412    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.295816 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4490c5-39e9-414e-9a46-944e6350b910-cni-binary-copy\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.295816 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-run-netns\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.295816 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-lib-modules\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.295816 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.295816 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-sysctl-d\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-cnibin\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-script-lib\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-systemd\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.295318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/821978bb-a169-482b-bfde-a70dcbc646d4-kubelet-config\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.295954    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:08.296175 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.296016    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:08.795998719 +0000 UTC m=+3.087473139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:08.296724 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.296205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4490c5-39e9-414e-9a46-944e6350b910-host-var-lib-kubelet\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.296724 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.296279    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovnkube-config\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.297048 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.297032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-tmp\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.298418 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.298399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e34a4577-5ec7-4373-8b1d-c6b212d224f9-agent-certs\") pod \"konnectivity-agent-n5vzf\" (UID: \"e34a4577-5ec7-4373-8b1d-c6b212d224f9\") " pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.298615 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.298568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-ovn-node-metrics-cert\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.298946 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.298929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-etc-tuned\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.304417 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.304148    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:08.304417 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.304172    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:08.304417 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.304186    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:08.304417 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.304250    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:08.804232358 +0000 UTC m=+3.095706789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:08.314312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.314291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmbch\" (UniqueName: \"kubernetes.io/projected/853ab9f9-27a9-4f31-8bcc-d3d9f49424d2-kube-api-access-pmbch\") pod \"tuned-nhglp\" (UID: \"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2\") " pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.314400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.314339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k2qs\" (UniqueName: \"kubernetes.io/projected/66c7d3f3-810a-4d7e-b0ac-88bc0257886d-kube-api-access-8k2qs\") pod \"ovnkube-node-622zx\" (UID: \"66c7d3f3-810a-4d7e-b0ac-88bc0257886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.314400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.314351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgbtd\" (UniqueName: \"kubernetes.io/projected/ab4490c5-39e9-414e-9a46-944e6350b910-kube-api-access-zgbtd\") pod \"multus-nz8zm\" (UID: \"ab4490c5-39e9-414e-9a46-944e6350b910\") " pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.314400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.314389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brc2s\" (UniqueName: \"kubernetes.io/projected/f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9-kube-api-access-brc2s\") pod \"multus-additional-cni-plugins-hvt4h\" (UID: \"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9\") " pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.384384 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.384353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8zvks"
Sep 11 04:40:08.391262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.391227    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2wtlv"
Sep 11 04:40:08.401930 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.401909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8"
Sep 11 04:40:08.413662 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.413637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-md27b"
Sep 11 04:40:08.420273 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.420259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvt4h"
Sep 11 04:40:08.426002 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.425981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nz8zm"
Sep 11 04:40:08.435534 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.435519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:08.443270 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.443248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:08.449018 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.448999    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nhglp"
Sep 11 04:40:08.537618 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.537568    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:40:08.694197 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.694112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:08.694361 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.694277    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:08.694361 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.694343    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:09.694327786 +0000 UTC m=+3.985802214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:08.895667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.895640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:08.895821 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:08.895691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:08.895821 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895796    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:08.895941 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895825    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:08.895941 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895842    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:08.895941 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895855    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:09.89584206 +0000 UTC m=+4.187316479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:08.895941 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895856    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:08.895941 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:08.895907    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:09.895891699 +0000 UTC m=+4.187366127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:09.121823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.121782    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 04:35:08 +0000 UTC" deadline="2028-03-25 05:07:27.142241697 +0000 UTC"
Sep 11 04:40:09.121823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.121820    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13464h27m18.020425802s"
Sep 11 04:40:09.186450 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.186426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:09.186611 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.186537    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:09.186611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.186595    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:09.186728 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.186694    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:09.216475 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.216446    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode669f686_df35_48bc_96dc_e7827a511c2f.slice/crio-e6f092ef757e9e212465c0044e77f4031472cdd0317148ff5f814eb382f51bf6 WatchSource:0}: Error finding container e6f092ef757e9e212465c0044e77f4031472cdd0317148ff5f814eb382f51bf6: Status 404 returned error can't find the container with id e6f092ef757e9e212465c0044e77f4031472cdd0317148ff5f814eb382f51bf6
Sep 11 04:40:09.217124 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.217103    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd524708_3d81_4da8_9e10_e8e4874a9a4e.slice/crio-cc978f94746f1c39ccb6879b6fb001091eb96f3fb69e5cdd27bdd5591c60a248 WatchSource:0}: Error finding container cc978f94746f1c39ccb6879b6fb001091eb96f3fb69e5cdd27bdd5591c60a248: Status 404 returned error can't find the container with id cc978f94746f1c39ccb6879b6fb001091eb96f3fb69e5cdd27bdd5591c60a248
Sep 11 04:40:09.218080 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.218024    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddecf3f4_042d_4f9f_b94e_c9deff6d68b0.slice/crio-6a09c9262d1144729096fdffb970ba05ca429d96c555ac20a46141db1b712c47 WatchSource:0}: Error finding container 6a09c9262d1144729096fdffb970ba05ca429d96c555ac20a46141db1b712c47: Status 404 returned error can't find the container with id 6a09c9262d1144729096fdffb970ba05ca429d96c555ac20a46141db1b712c47
Sep 11 04:40:09.220408 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.220384    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d35b7e_43b5_4f3c_bfb3_6958cbc04cc9.slice/crio-0281111b76214ad692035ac97be47bd2be09d0c7af79fb7bdddc00382266e1e4 WatchSource:0}: Error finding container 0281111b76214ad692035ac97be47bd2be09d0c7af79fb7bdddc00382266e1e4: Status 404 returned error can't find the container with id 0281111b76214ad692035ac97be47bd2be09d0c7af79fb7bdddc00382266e1e4
Sep 11 04:40:09.223732 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.223711    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4490c5_39e9_414e_9a46_944e6350b910.slice/crio-994ab92109f86fcca461bdc57556a7e75234be86250dd7289dcd1b4f5b770dac WatchSource:0}: Error finding container 994ab92109f86fcca461bdc57556a7e75234be86250dd7289dcd1b4f5b770dac: Status 404 returned error can't find the container with id 994ab92109f86fcca461bdc57556a7e75234be86250dd7289dcd1b4f5b770dac
Sep 11 04:40:09.224551 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.224532    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c7d3f3_810a_4d7e_b0ac_88bc0257886d.slice/crio-67ac8111bf6a60c997ac4aa07602353b482356527cc2bce3bb4f86cb789fca2b WatchSource:0}: Error finding container 67ac8111bf6a60c997ac4aa07602353b482356527cc2bce3bb4f86cb789fca2b: Status 404 returned error can't find the container with id 67ac8111bf6a60c997ac4aa07602353b482356527cc2bce3bb4f86cb789fca2b
Sep 11 04:40:09.225991 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.225912    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853ab9f9_27a9_4f31_8bcc_d3d9f49424d2.slice/crio-4493fec1d9b3f7477817855a7bc606756ba9514b0ee2732b590cbd9e65190a84 WatchSource:0}: Error finding container 4493fec1d9b3f7477817855a7bc606756ba9514b0ee2732b590cbd9e65190a84: Status 404 returned error can't find the container with id 4493fec1d9b3f7477817855a7bc606756ba9514b0ee2732b590cbd9e65190a84
Sep 11 04:40:09.227489 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.227471    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34a4577_5ec7_4373_8b1d_c6b212d224f9.slice/crio-e88ed12d1c10b589a0baa4bb6bad0f4077b877f200dc485e4aef811fa0c39174 WatchSource:0}: Error finding container e88ed12d1c10b589a0baa4bb6bad0f4077b877f200dc485e4aef811fa0c39174: Status 404 returned error can't find the container with id e88ed12d1c10b589a0baa4bb6bad0f4077b877f200dc485e4aef811fa0c39174
Sep 11 04:40:09.228140 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:09.228112    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6ab5b1_010f_470d_aed1_b35adf58e938.slice/crio-94bd716acfd4aa5c6abbeae6e82c003f81c85e50df08ed08e8e6da390b16eaa7 WatchSource:0}: Error finding container 94bd716acfd4aa5c6abbeae6e82c003f81c85e50df08ed08e8e6da390b16eaa7: Status 404 returned error can't find the container with id 94bd716acfd4aa5c6abbeae6e82c003f81c85e50df08ed08e8e6da390b16eaa7
Sep 11 04:40:09.701202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.701160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:09.701357 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.701316    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:09.701406 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.701397    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:11.701374923 +0000 UTC m=+5.992849355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:09.903646 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.903602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:09.903825 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:09.903664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:09.903825 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903783    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:09.903825 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903821    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:09.903975 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903843    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:09.903975 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903856    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:09.903975 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903858    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:11.903836927 +0000 UTC m=+6.195311362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:09.903975 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:09.903904    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:11.903888801 +0000 UTC m=+6.195363231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:10.124448 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.124241    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 04:35:08 +0000 UTC" deadline="2028-06-06 16:31:47.126330346 +0000 UTC"
Sep 11 04:40:10.124448 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.124279    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15227h51m37.002055703s"
Sep 11 04:40:10.189346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.189315    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:10.189507 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:10.189455    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:10.209269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.209235    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerStarted","Data":"0281111b76214ad692035ac97be47bd2be09d0c7af79fb7bdddc00382266e1e4"}
Sep 11 04:40:10.214514 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.214487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8" event={"ID":"bd524708-3d81-4da8-9e10-e8e4874a9a4e","Type":"ContainerStarted","Data":"cc978f94746f1c39ccb6879b6fb001091eb96f3fb69e5cdd27bdd5591c60a248"}
Sep 11 04:40:10.223145 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.223108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2wtlv" event={"ID":"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0","Type":"ContainerStarted","Data":"6a09c9262d1144729096fdffb970ba05ca429d96c555ac20a46141db1b712c47"}
Sep 11 04:40:10.229539 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.229515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n5vzf" event={"ID":"e34a4577-5ec7-4373-8b1d-c6b212d224f9","Type":"ContainerStarted","Data":"e88ed12d1c10b589a0baa4bb6bad0f4077b877f200dc485e4aef811fa0c39174"}
Sep 11 04:40:10.235526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.235465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nhglp" event={"ID":"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2","Type":"ContainerStarted","Data":"4493fec1d9b3f7477817855a7bc606756ba9514b0ee2732b590cbd9e65190a84"}
Sep 11 04:40:10.238417 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.238355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8zvks" event={"ID":"e669f686-df35-48bc-96dc-e7827a511c2f","Type":"ContainerStarted","Data":"e6f092ef757e9e212465c0044e77f4031472cdd0317148ff5f814eb382f51bf6"}
Sep 11 04:40:10.270628 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.270245    2581 generic.go:356] "Generic (PLEG): container finished" podID="0f0cab04a8f401f88ebf3b4b63e59fc8" containerID="14fcdfa01bc8a1e60a830d549a54e6c4e4bdefa244f257ab21177e15138c9c2e" exitCode=0
Sep 11 04:40:10.270628 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.270376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal" event={"ID":"0f0cab04a8f401f88ebf3b4b63e59fc8","Type":"ContainerDied","Data":"14fcdfa01bc8a1e60a830d549a54e6c4e4bdefa244f257ab21177e15138c9c2e"}
Sep 11 04:40:10.288301 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.288277    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal" event={"ID":"4e4442bfe415e79ad545fffb4094f327","Type":"ContainerStarted","Data":"05e2bc872753fd4a0a963bb5feddf60ffec94ef722a24e13badc16a87cc08dca"}
Sep 11 04:40:10.301195 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.301171    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-md27b" event={"ID":"dd6ab5b1-010f-470d-aed1-b35adf58e938","Type":"ContainerStarted","Data":"94bd716acfd4aa5c6abbeae6e82c003f81c85e50df08ed08e8e6da390b16eaa7"}
Sep 11 04:40:10.310626 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.310569    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"67ac8111bf6a60c997ac4aa07602353b482356527cc2bce3bb4f86cb789fca2b"}
Sep 11 04:40:10.312258 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:10.312232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nz8zm" event={"ID":"ab4490c5-39e9-414e-9a46-944e6350b910","Type":"ContainerStarted","Data":"994ab92109f86fcca461bdc57556a7e75234be86250dd7289dcd1b4f5b770dac"}
Sep 11 04:40:11.186942 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.186910    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:11.187513 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.187030    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:11.187513 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.187362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:11.187513 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.187468    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:11.358738 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.358698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal" event={"ID":"0f0cab04a8f401f88ebf3b4b63e59fc8","Type":"ContainerStarted","Data":"8374901af4a1f1eab6f0839a7cb851bfffe753d86e685ea303808d3f39805f46"}
Sep 11 04:40:11.377345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.377292    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-137.ec2.internal" podStartSLOduration=4.377273852 podStartE2EDuration="4.377273852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:10.303126517 +0000 UTC m=+4.594600955" watchObservedRunningTime="2026-09-11 04:40:11.377273852 +0000 UTC m=+5.668748294"
Sep 11 04:40:11.722523 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.722485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:11.722710 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.722683    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:11.722775 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.722744    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:15.722725906 +0000 UTC m=+10.014200340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:11.924726 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.924563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:11.924726 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:11.924677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:11.924966 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.924806    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:11.924966 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.924886    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:15.92486792 +0000 UTC m=+10.216342340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:11.924966 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.924956    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:11.925138 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.924979    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:11.925138 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.924992    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:11.925138 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:11.925042    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:15.925025859 +0000 UTC m=+10.216500276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:12.187881 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:12.187812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:12.188302 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:12.187944    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:13.187091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:13.187051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:13.187274 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:13.187173    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:13.187552 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:13.187535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:13.187697 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:13.187678    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:14.186310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:14.186281    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:14.186777 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:14.186401    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:15.186692 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:15.186614    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:15.186692 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:15.186655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:15.187111 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.186740    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:15.187111 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.186884    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:15.755477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:15.754918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:15.755477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.755088    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:15.755477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.755152    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:23.755133919 +0000 UTC m=+18.046608336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:15.956977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:15.957026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957162    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957181    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957195    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957253    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:23.957233319 +0000 UTC m=+18.248707753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957659    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:15.957743 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:15.957709    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:23.957694706 +0000 UTC m=+18.249169121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:16.187787 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:16.187718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:16.188130 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:16.187801    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:17.186161 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:17.186127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:17.186345 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:17.186254    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:17.186634 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:17.186612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:17.186754 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:17.186733    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:18.186519 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:18.186169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:18.186519 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:18.186307    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:19.186947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:19.186914    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:19.186947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:19.186955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:19.187443 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:19.187052    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:19.187443 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:19.187190    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:20.187123 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:20.186870    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:20.187549 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:20.187161    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:21.186631 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:21.186599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:21.186819 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:21.186704    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:21.187060 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:21.187043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:21.187177 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:21.187143    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:22.187030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:22.186997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:22.187211 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:22.187120    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:23.186132 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:23.186099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:23.186132 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:23.186127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:23.186407 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:23.186224    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:23.186407 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:23.186369    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:23.812913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:23.812875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:23.813357 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:23.813325    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:23.813422 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:23.813409    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:39.813384697 +0000 UTC m=+34.104859131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:24.014201 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:24.014167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:24.014368 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:24.014217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:24.014368 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014316    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:24.014368 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014332    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:24.014368 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014351    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:24.014368 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014362    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:24.014547 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014381    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:40.014362852 +0000 UTC m=+34.305837290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:24.014547 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.014409    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:40.014393772 +0000 UTC m=+34.305868207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:24.186472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:24.186393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:24.186634 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:24.186510    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:25.186372 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:25.186339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:25.186841 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:25.186456    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:25.186841 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:25.186516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:25.186841 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:25.186647    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:26.186831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:26.186802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:26.187287 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:26.186887    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:27.186324 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:27.186298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:27.186428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:27.186300    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:27.186481 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:27.186465    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:27.186520 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:27.186385    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:27.401305 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:27.400533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nhglp" event={"ID":"853ab9f9-27a9-4f31-8bcc-d3d9f49424d2","Type":"ContainerStarted","Data":"5aad8ea4732f782d595ae25361ec444dfd87420c9d59228be0e1744ec8426902"}
Sep 11 04:40:27.416887 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:27.416850    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-137.ec2.internal" podStartSLOduration=20.416835229 podStartE2EDuration="20.416835229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:11.378480869 +0000 UTC m=+5.669955306" watchObservedRunningTime="2026-09-11 04:40:27.416835229 +0000 UTC m=+21.708309666"
Sep 11 04:40:28.186731 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.186547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:28.186933 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:28.186813    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:28.404371 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.404333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-md27b" event={"ID":"dd6ab5b1-010f-470d-aed1-b35adf58e938","Type":"ContainerStarted","Data":"91f68517b6866b1b10d24d76c5ba6c4c306a294c0435a40b7f64f84723331f71"}
Sep 11 04:40:28.407718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"50eac67290af677c98b6e2c0ffdea3b73c5d1a6efc1949c11f5312bb0b737a79"}
Sep 11 04:40:28.407859 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"a87a1e4a866b5d19702867a5508e91b205d9adfae987ab1889b896d90fdcdec4"}
Sep 11 04:40:28.407859 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"f6d2402184dc37f26fa24a39440f6910694fbcdfac96d970bcf9fa895743877c"}
Sep 11 04:40:28.407859 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"e94db3b461d35e67d07a7af0a09b263486ddfeed10e2d703374410cc11104643"}
Sep 11 04:40:28.408001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"6aaf5f8860bca5cba3eb5559d8c048ea8f5a8ee3d2dd986c0e2219be7ca0e5fa"}
Sep 11 04:40:28.408001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.407921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"30716544607b6d4f4ef6097d35e19210d8b3366604feaffd1126e00300a0869f"}
Sep 11 04:40:28.409421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.409392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nz8zm" event={"ID":"ab4490c5-39e9-414e-9a46-944e6350b910","Type":"ContainerStarted","Data":"ad736bf262500e850bfbff2b6b9caf795f1dd0e7f789f1468ee4bab2c92283dd"}
Sep 11 04:40:28.411380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.411358    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="32dbf34bb655774667f373a96b133ecefdb5a35dad1bc8544954fc7e10c26d39" exitCode=0
Sep 11 04:40:28.411498 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.411425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"32dbf34bb655774667f373a96b133ecefdb5a35dad1bc8544954fc7e10c26d39"}
Sep 11 04:40:28.413536 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.413511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8" event={"ID":"bd524708-3d81-4da8-9e10-e8e4874a9a4e","Type":"ContainerStarted","Data":"86b40caa9a3dc30acb39979792adfd016d887ae04f399e8f651c41b02c656ed1"}
Sep 11 04:40:28.417337 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.416733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2wtlv" event={"ID":"ddecf3f4-042d-4f9f-b94e-c9deff6d68b0","Type":"ContainerStarted","Data":"3cafe81c83c47f2a54d4238c6e50faf19da1b276e624d661f5853be7c62e83d6"}
Sep 11 04:40:28.419499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.419477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n5vzf" event={"ID":"e34a4577-5ec7-4373-8b1d-c6b212d224f9","Type":"ContainerStarted","Data":"d780511a5376c4b91459934674db61ced534ac28f9a89a5cd500e8670e297dab"}
Sep 11 04:40:28.422220 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.422177    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-md27b" podStartSLOduration=4.431266619 podStartE2EDuration="22.422161232s" totalImagesPullingTime="17.990894613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:28.42170863 +0000 UTC m=+22.713183108" watchObservedRunningTime="2026-09-11 04:40:28.422161232 +0000 UTC m=+22.713635698"
Sep 11 04:40:28.422736 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.422707    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nhglp" podStartSLOduration=4.428060394 podStartE2EDuration="22.422699113s" totalImagesPullingTime="17.994638719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:27.416755877 +0000 UTC m=+21.708230314" watchObservedRunningTime="2026-09-11 04:40:28.422699113 +0000 UTC m=+22.714173552"
Sep 11 04:40:28.453856 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.453819    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nz8zm" podStartSLOduration=4.405928003 podStartE2EDuration="22.453807204s" totalImagesPullingTime="18.047879201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:28.453416287 +0000 UTC m=+22.744890724" watchObservedRunningTime="2026-09-11 04:40:28.453807204 +0000 UTC m=+22.745281640"
Sep 11 04:40:28.467046 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.467013    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n5vzf" podStartSLOduration=11.900738859 podStartE2EDuration="22.467002273s" totalImagesPullingTime="10.566263414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:28.466860917 +0000 UTC m=+22.758335354" watchObservedRunningTime="2026-09-11 04:40:28.467002273 +0000 UTC m=+22.758476710"
Sep 11 04:40:28.883951 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:28.883810    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 04:40:29.153054 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.152954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:29.154015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.153991    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:29.163908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.163816    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T04:40:28.88394755Z","UUID":"501002cf-a399-404d-99f8-85c169a83ad7","Handler":null,"Name":"","Endpoint":""}
Sep 11 04:40:29.165796 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.165777    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 04:40:29.165911 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.165806    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 04:40:29.168530 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.168485    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2wtlv" podStartSLOduration=5.187880262 podStartE2EDuration="23.168465719s" totalImagesPullingTime="17.980585457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:28.481165313 +0000 UTC m=+22.772639750" watchObservedRunningTime="2026-09-11 04:40:29.168465719 +0000 UTC m=+23.459940157"
Sep 11 04:40:29.187140 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.187120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:29.187244 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:29.187223    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:29.187301 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.187255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:29.187359 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:29.187338    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:29.424051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.423974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8zvks" event={"ID":"e669f686-df35-48bc-96dc-e7827a511c2f","Type":"ContainerStarted","Data":"6d2269ff1a40475d90647b43e90ae29801968a1480cb7fbe958c22c3a5dfa7dd"}
Sep 11 04:40:29.426453 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.426425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8" event={"ID":"bd524708-3d81-4da8-9e10-e8e4874a9a4e","Type":"ContainerStarted","Data":"f225aced1f5c41f65a0d18bc3f141d758e632978385b2a892a27a4419a57b6f6"}
Sep 11 04:40:29.427047 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.427024    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:29.427493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.427474    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n5vzf"
Sep 11 04:40:29.437368 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:29.437334    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8zvks" podStartSLOduration=5.434217205 podStartE2EDuration="23.437317587s" totalImagesPullingTime="18.003100382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:29.43709075 +0000 UTC m=+23.728565184" watchObservedRunningTime="2026-09-11 04:40:29.437317587 +0000 UTC m=+23.728792025"
Sep 11 04:40:30.186605 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:30.186501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:30.186759 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:30.186662    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:30.433083 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:30.433042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"208201471aab18d660397985971517648155925551f2686c6de361291ab0baf3"}
Sep 11 04:40:30.435397 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:30.435367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8" event={"ID":"bd524708-3d81-4da8-9e10-e8e4874a9a4e","Type":"ContainerStarted","Data":"b0dffcd1274f7aedf93ae18db8f3917118d768adf4e69d4fba47622599cecfb1"}
Sep 11 04:40:30.451283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:30.451233    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxnv8" podStartSLOduration=4.111085193 podStartE2EDuration="24.451219544s" totalImagesPullingTime="20.340134351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:30.450705885 +0000 UTC m=+24.742180322" watchObservedRunningTime="2026-09-11 04:40:30.451219544 +0000 UTC m=+24.742694001"
Sep 11 04:40:31.186394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:31.186363    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:31.186573 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:31.186458    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:31.186573 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:31.186491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:31.186573 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:31.186545    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:32.186981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:32.186954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:32.187371 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:32.187065    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:33.186726 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.186696    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:33.186871 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:33.186813    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:33.186871 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.186862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:33.186960 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:33.186943    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:33.450609 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.447470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" event={"ID":"66c7d3f3-810a-4d7e-b0ac-88bc0257886d","Type":"ContainerStarted","Data":"973c8e5b3447a0b216f72a6b2bd3160ab633aef414df04f05b2ce15aeb376169"}
Sep 11 04:40:33.450609 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.449879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:33.450609 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.449977    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:33.450609 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.450047    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:33.468032 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.468011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:33.469873 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.469854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:40:33.475294 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:33.475260    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-622zx" podStartSLOduration=9.422461178 podStartE2EDuration="27.47524805s" totalImagesPullingTime="18.052786872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:33.475140705 +0000 UTC m=+27.766615141" watchObservedRunningTime="2026-09-11 04:40:33.47524805 +0000 UTC m=+27.766722487"
Sep 11 04:40:34.187067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:34.187038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:34.187228 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:34.187152    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:34.451017 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:34.450986    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="d33c3e15a42ea63228a10e9c5361f26d0335bd5ccffd24bac4529465cc7923cf" exitCode=0
Sep 11 04:40:34.451460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:34.451068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"d33c3e15a42ea63228a10e9c5361f26d0335bd5ccffd24bac4529465cc7923cf"}
Sep 11 04:40:35.186835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.186760    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:35.186976 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:35.186871    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:35.186976 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.186936    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:35.187078 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:35.187056    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:35.190846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.190818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-psmdc"]
Sep 11 04:40:35.194487 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.194461    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72h62"]
Sep 11 04:40:35.194639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.194530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:35.194639 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:35.194618    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:35.195758 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.195552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hqzh"]
Sep 11 04:40:35.455659 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.455631    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="b0577b4bd801bc6f9b54aa4fbd2ffd5ded59e30320f4c1f145e4cfc1f6746229" exitCode=0
Sep 11 04:40:35.456236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.455719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"b0577b4bd801bc6f9b54aa4fbd2ffd5ded59e30320f4c1f145e4cfc1f6746229"}
Sep 11 04:40:35.456236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.455863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:35.456622 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:35.456237    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:35.456764 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:35.456643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:35.457413 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:35.457388    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:36.461819 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:36.461622    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="8e6ac59edf145ec56f55bb5aa281aaa6e57e37d54038be882e7ef276f69f04dc" exitCode=0
Sep 11 04:40:36.462243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:36.461715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"8e6ac59edf145ec56f55bb5aa281aaa6e57e37d54038be882e7ef276f69f04dc"}
Sep 11 04:40:37.186987 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:37.186900    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:37.186987 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:37.186963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:37.187190 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:37.187082    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:37.187190 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:37.187122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:37.187298 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:37.187271    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:37.187349 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:37.187329    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:39.186729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:39.186698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:39.187169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:39.186801    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:39.187169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:39.186896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:39.187169 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:39.186912    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-psmdc" podUID="bca39b9b-29ed-46e8-8396-bec1d86aae18"
Sep 11 04:40:39.187169 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:39.186975    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hqzh" podUID="66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7"
Sep 11 04:40:39.187169 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:39.187034    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72h62" podUID="821978bb-a169-482b-bfde-a70dcbc646d4"
Sep 11 04:40:39.832707 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:39.832679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:39.832868 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:39.832789    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:39.832868 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:39.832860    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.832839582 +0000 UTC m=+66.124314034 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:40.034400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.034312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:40.034400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.034364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034485    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034485    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034509    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034524    2581 projected.go:196] Error preparing data for projected volume kube-api-access-rcfgd for pod openshift-network-diagnostics/network-check-target-psmdc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034563    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret podName:821978bb-a169-482b-bfde-a70dcbc646d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:12.034541363 +0000 UTC m=+66.326015797 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret") pod "global-pull-secret-syncer-72h62" (UID: "821978bb-a169-482b-bfde-a70dcbc646d4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:40.034721 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.034602    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd podName:bca39b9b-29ed-46e8-8396-bec1d86aae18 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:12.034574191 +0000 UTC m=+66.326048608 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcfgd" (UniqueName: "kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd") pod "network-check-target-psmdc" (UID: "bca39b9b-29ed-46e8-8396-bec1d86aae18") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:40.040218 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.040197    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-137.ec2.internal" event="NodeReady"
Sep 11 04:40:40.040343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.040333    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 04:40:40.085228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.085194    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-42vp6"]
Sep 11 04:40:40.103518 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.103486    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xtll9"]
Sep 11 04:40:40.103703 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.103600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.108619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.108571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 04:40:40.108751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.108728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 04:40:40.108843 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.108730    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v459k\""
Sep 11 04:40:40.114856 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.114837    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-42vp6"]
Sep 11 04:40:40.114979 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.114868    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xtll9"]
Sep 11 04:40:40.114979 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.114961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.117189 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.117171    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 04:40:40.117313 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.117205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 04:40:40.117552 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.117535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 04:40:40.117887 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.117870    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6kvx7\""
Sep 11 04:40:40.235186 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmgs4\" (UniqueName: \"kubernetes.io/projected/a78ba00e-4e12-4712-9f78-408d47756b56-kube-api-access-gmgs4\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.235569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa39515-1242-4d0e-a11d-4218cfa55fb9-cert\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.235569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78ba00e-4e12-4712-9f78-408d47756b56-config-volume\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.235569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78ba00e-4e12-4712-9f78-408d47756b56-tmp-dir\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.235569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.235569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.235387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfjwk\" (UniqueName: \"kubernetes.io/projected/6fa39515-1242-4d0e-a11d-4218cfa55fb9-kube-api-access-hfjwk\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.336527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.336791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfjwk\" (UniqueName: \"kubernetes.io/projected/6fa39515-1242-4d0e-a11d-4218cfa55fb9-kube-api-access-hfjwk\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.336791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmgs4\" (UniqueName: \"kubernetes.io/projected/a78ba00e-4e12-4712-9f78-408d47756b56-kube-api-access-gmgs4\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.336791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa39515-1242-4d0e-a11d-4218cfa55fb9-cert\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.336791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.336629    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:40.336791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.336700    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:40.836681648 +0000 UTC m=+35.128156078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:40.336998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78ba00e-4e12-4712-9f78-408d47756b56-config-volume\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.336998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.336866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78ba00e-4e12-4712-9f78-408d47756b56-tmp-dir\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.337176 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.337156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78ba00e-4e12-4712-9f78-408d47756b56-config-volume\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.337232 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.337165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78ba00e-4e12-4712-9f78-408d47756b56-tmp-dir\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.341781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.341759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa39515-1242-4d0e-a11d-4218cfa55fb9-cert\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.344720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.344696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmgs4\" (UniqueName: \"kubernetes.io/projected/a78ba00e-4e12-4712-9f78-408d47756b56-kube-api-access-gmgs4\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.345030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.345010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfjwk\" (UniqueName: \"kubernetes.io/projected/6fa39515-1242-4d0e-a11d-4218cfa55fb9-kube-api-access-hfjwk\") pod \"ingress-canary-xtll9\" (UID: \"6fa39515-1242-4d0e-a11d-4218cfa55fb9\") " pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.426996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.426958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xtll9"
Sep 11 04:40:40.839342 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:40.839302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:40.839508 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.839444    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:40.839508 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:40.839502    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:41.839488621 +0000 UTC m=+36.130963035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:41.186551 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.186342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:40:41.186693 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.186369    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:40:41.186693 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.186397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:40:41.189543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189519    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 04:40:41.189543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189534    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 04:40:41.189749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189553    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 04:40:41.189749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189529    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kl86\""
Sep 11 04:40:41.189749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 04:40:41.189749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.189534    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rs4qk\""
Sep 11 04:40:41.271627 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.271600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xtll9"]
Sep 11 04:40:41.275092 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:40:41.275066    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa39515_1242_4d0e_a11d_4218cfa55fb9.slice/crio-58468ffd76a2a55b3ff6681503f78da2bcd98b1c30e6e1827d0565b09826718d WatchSource:0}: Error finding container 58468ffd76a2a55b3ff6681503f78da2bcd98b1c30e6e1827d0565b09826718d: Status 404 returned error can't find the container with id 58468ffd76a2a55b3ff6681503f78da2bcd98b1c30e6e1827d0565b09826718d
Sep 11 04:40:41.475345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.475315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xtll9" event={"ID":"6fa39515-1242-4d0e-a11d-4218cfa55fb9","Type":"ContainerStarted","Data":"58468ffd76a2a55b3ff6681503f78da2bcd98b1c30e6e1827d0565b09826718d"}
Sep 11 04:40:41.478302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.478271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerStarted","Data":"55a1f560e59ef64736737dad85e6c98c69a15dbcfe52599b7aef1b0fc5a90585"}
Sep 11 04:40:41.849763 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:41.849675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:41.849914 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:41.849862    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:41.849952 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:41.849935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:43.849914255 +0000 UTC m=+38.141388689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:42.484124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:42.484089    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="55a1f560e59ef64736737dad85e6c98c69a15dbcfe52599b7aef1b0fc5a90585" exitCode=0
Sep 11 04:40:42.484710 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:42.484145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"55a1f560e59ef64736737dad85e6c98c69a15dbcfe52599b7aef1b0fc5a90585"}
Sep 11 04:40:43.490322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:43.490286    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9" containerID="e5db92d8cdcad60f92c460a7698c572a45580b98222189e69ab43d734105f521" exitCode=0
Sep 11 04:40:43.490927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:43.490340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerDied","Data":"e5db92d8cdcad60f92c460a7698c572a45580b98222189e69ab43d734105f521"}
Sep 11 04:40:43.865976 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:43.865948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:43.866079 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:43.866061    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:43.866148 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:43.866106    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:47.866093533 +0000 UTC m=+42.157567951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:44.493894 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:44.493854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xtll9" event={"ID":"6fa39515-1242-4d0e-a11d-4218cfa55fb9","Type":"ContainerStarted","Data":"7a725763e80e0c954bcf9e77bbd6a8862d513fe64eb7e8eacae7e71950e53738"}
Sep 11 04:40:44.499322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:44.499292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" event={"ID":"f8d35b7e-43b5-4f3c-bfb3-6958cbc04cc9","Type":"ContainerStarted","Data":"db0c388ced124320fe36cc5447186d170dd073b135661eb530e382a21eb825de"}
Sep 11 04:40:44.509455 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:44.509415    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xtll9" podStartSLOduration=1.819336445 podStartE2EDuration="4.509402736s" totalImagesPullingTime="2.690066291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:44.508513718 +0000 UTC m=+38.799988155" watchObservedRunningTime="2026-09-11 04:40:44.509402736 +0000 UTC m=+38.800877216"
Sep 11 04:40:44.533274 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:44.533219    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hvt4h" podStartSLOduration=9.217483983 podStartE2EDuration="38.533200089s" totalImagesPullingTime="29.315716106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:44.531756683 +0000 UTC m=+38.823231120" watchObservedRunningTime="2026-09-11 04:40:44.533200089 +0000 UTC m=+38.824674526"
Sep 11 04:40:47.902615 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:47.902348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:47.902989 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:47.902504    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:47.902989 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:47.902743    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:55.902727534 +0000 UTC m=+50.194201953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:55.957547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:40:55.957506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:40:55.958028 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:55.957659    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:55.958028 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:40:55.957735    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.957715049 +0000 UTC m=+66.249189479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:41:04.504096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.504058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"]
Sep 11 04:41:04.537225 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.537193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"]
Sep 11 04:41:04.537354 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.537292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.539866 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.539834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 04:41:04.540001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.539941    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:41:04.540710 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.540693    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qkxmj\""
Sep 11 04:41:04.540791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.540762    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 04:41:04.540858 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.540814    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 04:41:04.602774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.602746    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"]
Sep 11 04:41:04.614289 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.614261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7a4371a-2787-4d9f-870e-e7c137a75ed7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.614385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.614315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a4371a-2787-4d9f-870e-e7c137a75ed7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.614444 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.614384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpqp2\" (UniqueName: \"kubernetes.io/projected/a7a4371a-2787-4d9f-870e-e7c137a75ed7-kube-api-access-xpqp2\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.627981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.627961    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"]
Sep 11 04:41:04.628095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.628080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.630338 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.630321    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 04:41:04.631882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.631864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 04:41:04.631882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.631876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 04:41:04.632018 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.631866    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 04:41:04.632018 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.631918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6ghxt\""
Sep 11 04:41:04.646089 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.646052    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-jm555"]
Sep 11 04:41:04.646206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.646189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"
Sep 11 04:41:04.648167 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.648151    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:41:04.648300 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.648284    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kpxqh\""
Sep 11 04:41:04.648350 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.648323    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 04:41:04.659525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.659508    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"]
Sep 11 04:41:04.659611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.659533    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"]
Sep 11 04:41:04.659611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.659550    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jm555"]
Sep 11 04:41:04.659683 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.659630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.662481 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.662462    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 04:41:04.662594 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.662566    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 04:41:04.663146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.663131    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 04:41:04.663216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.663130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-q6drg\""
Sep 11 04:41:04.663216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.663137    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 04:41:04.671121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.671100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 04:41:04.709593 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.709551    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-tzsh8"]
Sep 11 04:41:04.715236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbj6\" (UniqueName: \"kubernetes.io/projected/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-kube-api-access-gbbj6\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.715309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.715309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-tmp\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-serving-cert\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-snapshots\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7a4371a-2787-4d9f-870e-e7c137a75ed7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.715430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a4371a-2787-4d9f-870e-e7c137a75ed7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.715430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.715430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4kl8\" (UniqueName: \"kubernetes.io/projected/276dc052-1ce1-4884-ae66-788ebe3d5fd8-kube-api-access-x4kl8\") pod \"volume-data-source-validator-86695b7cc7-kb9gp\" (UID: \"276dc052-1ce1-4884-ae66-788ebe3d5fd8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"
Sep 11 04:41:04.715613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpqp2\" (UniqueName: \"kubernetes.io/projected/a7a4371a-2787-4d9f-870e-e7c137a75ed7-kube-api-access-xpqp2\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.715613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-service-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5wd\" (UniqueName: \"kubernetes.io/projected/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-kube-api-access-vw5wd\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.715922 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.715902    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7a4371a-2787-4d9f-870e-e7c137a75ed7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.718671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.718654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a4371a-2787-4d9f-870e-e7c137a75ed7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.728078 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.728060    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:41:04.728230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.728168    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.731124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.731078    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dl4g4\""
Sep 11 04:41:04.731235 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.731144    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 04:41:04.731294 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.731282    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:41:04.731784 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.731764    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 04:41:04.731882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.731768    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 04:41:04.735475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.735459    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 04:41:04.738657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.738637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpqp2\" (UniqueName: \"kubernetes.io/projected/a7a4371a-2787-4d9f-870e-e7c137a75ed7-kube-api-access-xpqp2\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7mqwk\" (UID: \"a7a4371a-2787-4d9f-870e-e7c137a75ed7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.742382 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.742365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-tzsh8"]
Sep 11 04:41:04.742448 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.742396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:41:04.742492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.742456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.744525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.744499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 04:41:04.744700 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.744686    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 04:41:04.744791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.744690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkc4\""
Sep 11 04:41:04.745266 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.745252    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 04:41:04.750296 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.750279    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 04:41:04.816617 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbj6\" (UniqueName: \"kubernetes.io/projected/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-kube-api-access-gbbj6\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.816617 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jswm8\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.816735 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.816785 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.816785 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4kl8\" (UniqueName: \"kubernetes.io/projected/276dc052-1ce1-4884-ae66-788ebe3d5fd8-kube-api-access-x4kl8\") pod \"volume-data-source-validator-86695b7cc7-kb9gp\" (UID: \"276dc052-1ce1-4884-ae66-788ebe3d5fd8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c127bc70-006d-4bf8-adab-0d937a3b5102-serving-cert\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-tmp\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-serving-cert\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.816888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-snapshots\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:04.816920    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:04.817029    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:05.317009633 +0000 UTC m=+59.608484068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.816955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-trusted-ca\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.817170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-service-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817648 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.817648 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fzjg\" (UniqueName: \"kubernetes.io/projected/c127bc70-006d-4bf8-adab-0d937a3b5102-kube-api-access-5fzjg\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.817648 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-config\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.817648 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5wd\" (UniqueName: \"kubernetes.io/projected/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-kube-api-access-vw5wd\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817648 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-tmp\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-snapshots\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.817912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.817858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-service-ca-bundle\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.818082 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.818061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.819288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.819267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-serving-cert\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.825213 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.825189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5wd\" (UniqueName: \"kubernetes.io/projected/364e817b-78a5-4e76-8fb8-6ac3644bbe8b-kube-api-access-vw5wd\") pod \"insights-operator-5844b856db-jm555\" (UID: \"364e817b-78a5-4e76-8fb8-6ac3644bbe8b\") " pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:04.825284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.825221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbj6\" (UniqueName: \"kubernetes.io/projected/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-kube-api-access-gbbj6\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:04.825705 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.825685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4kl8\" (UniqueName: \"kubernetes.io/projected/276dc052-1ce1-4884-ae66-788ebe3d5fd8-kube-api-access-x4kl8\") pod \"volume-data-source-validator-86695b7cc7-kb9gp\" (UID: \"276dc052-1ce1-4884-ae66-788ebe3d5fd8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"
Sep 11 04:41:04.846255 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.846238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"
Sep 11 04:41:04.918642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jswm8\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c127bc70-006d-4bf8-adab-0d937a3b5102-serving-cert\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-trusted-ca\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.918973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzjg\" (UniqueName: \"kubernetes.io/projected/c127bc70-006d-4bf8-adab-0d937a3b5102-kube-api-access-5fzjg\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.919007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-config\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.919265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:04.919724    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:04.919741    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd7c84f5-2d5rw: secret "image-registry-tls" not found
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:04.919810    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls podName:a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:05.419791161 +0000 UTC m=+59.711265592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls") pod "image-registry-85fd7c84f5-2d5rw" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4") : secret "image-registry-tls" not found
Sep 11 04:41:04.921624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.919881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-config\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.922800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.919946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.922800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.920645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c127bc70-006d-4bf8-adab-0d937a3b5102-trusted-ca\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.922800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.920946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.922800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.922159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c127bc70-006d-4bf8-adab-0d937a3b5102-serving-cert\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.923817 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.923710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.924325 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.924300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.928026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.928002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jswm8\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.932223 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.932196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:04.932568 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.932549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fzjg\" (UniqueName: \"kubernetes.io/projected/c127bc70-006d-4bf8-adab-0d937a3b5102-kube-api-access-5fzjg\") pod \"console-operator-696884fb9c-tzsh8\" (UID: \"c127bc70-006d-4bf8-adab-0d937a3b5102\") " pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:04.954485 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.954467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"
Sep 11 04:41:04.967658 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:04.967639    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jm555"
Sep 11 04:41:05.039964 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.039351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:05.142146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.142125    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk"]
Sep 11 04:41:05.145854 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:05.145812    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a4371a_2787_4d9f_870e_e7c137a75ed7.slice/crio-6dec5ed3fdc12b2dfb2609ebcac49a1746f3ee3f499e42609472619bd39382bd WatchSource:0}: Error finding container 6dec5ed3fdc12b2dfb2609ebcac49a1746f3ee3f499e42609472619bd39382bd: Status 404 returned error can't find the container with id 6dec5ed3fdc12b2dfb2609ebcac49a1746f3ee3f499e42609472619bd39382bd
Sep 11 04:41:05.248460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.248438    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp"]
Sep 11 04:41:05.250772 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:05.250745    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod276dc052_1ce1_4884_ae66_788ebe3d5fd8.slice/crio-c856e579df1344d96659a3193cb0269dadabb8c0a71d22b36527a7259e70a2b8 WatchSource:0}: Error finding container c856e579df1344d96659a3193cb0269dadabb8c0a71d22b36527a7259e70a2b8: Status 404 returned error can't find the container with id c856e579df1344d96659a3193cb0269dadabb8c0a71d22b36527a7259e70a2b8
Sep 11 04:41:05.273359 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.273330    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jm555"]
Sep 11 04:41:05.276115 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:05.276092    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364e817b_78a5_4e76_8fb8_6ac3644bbe8b.slice/crio-b1eba2cc1ba2762b515a638fa100a38d834d386bac176808f07b1352826fea87 WatchSource:0}: Error finding container b1eba2cc1ba2762b515a638fa100a38d834d386bac176808f07b1352826fea87: Status 404 returned error can't find the container with id b1eba2cc1ba2762b515a638fa100a38d834d386bac176808f07b1352826fea87
Sep 11 04:41:05.318820 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.318798    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-tzsh8"]
Sep 11 04:41:05.321728 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:05.321704    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc127bc70_006d_4bf8_adab_0d937a3b5102.slice/crio-9263cdb61fcfe44d57c8a36f5aa714d2674ff2b732e7b1bbec8ef0d3e5c637a2 WatchSource:0}: Error finding container 9263cdb61fcfe44d57c8a36f5aa714d2674ff2b732e7b1bbec8ef0d3e5c637a2: Status 404 returned error can't find the container with id 9263cdb61fcfe44d57c8a36f5aa714d2674ff2b732e7b1bbec8ef0d3e5c637a2
Sep 11 04:41:05.321896 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.321876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:05.322014 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:05.322002    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:05.322060 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:05.322054    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:06.322039917 +0000 UTC m=+60.613514336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:05.422541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.422475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:05.422679 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:05.422639    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:41:05.422679 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:05.422658    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd7c84f5-2d5rw: secret "image-registry-tls" not found
Sep 11 04:41:05.422760 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:05.422723    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls podName:a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:06.422706416 +0000 UTC m=+60.714180832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls") pod "image-registry-85fd7c84f5-2d5rw" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4") : secret "image-registry-tls" not found
Sep 11 04:41:05.474268 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.474239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-622zx"
Sep 11 04:41:05.552949 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.552919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" event={"ID":"c127bc70-006d-4bf8-adab-0d937a3b5102","Type":"ContainerStarted","Data":"9263cdb61fcfe44d57c8a36f5aa714d2674ff2b732e7b1bbec8ef0d3e5c637a2"}
Sep 11 04:41:05.554421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.554396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jm555" event={"ID":"364e817b-78a5-4e76-8fb8-6ac3644bbe8b","Type":"ContainerStarted","Data":"b1eba2cc1ba2762b515a638fa100a38d834d386bac176808f07b1352826fea87"}
Sep 11 04:41:05.555619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.555600    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp" event={"ID":"276dc052-1ce1-4884-ae66-788ebe3d5fd8","Type":"ContainerStarted","Data":"c856e579df1344d96659a3193cb0269dadabb8c0a71d22b36527a7259e70a2b8"}
Sep 11 04:41:05.556614 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:05.556596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk" event={"ID":"a7a4371a-2787-4d9f-870e-e7c137a75ed7","Type":"ContainerStarted","Data":"6dec5ed3fdc12b2dfb2609ebcac49a1746f3ee3f499e42609472619bd39382bd"}
Sep 11 04:41:06.331266 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:06.331228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:06.331491 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:06.331473    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:06.331559 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:06.331547    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:08.331528576 +0000 UTC m=+62.623003006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:06.431927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:06.431892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:06.432096 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:06.432050    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:41:06.432096 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:06.432064    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd7c84f5-2d5rw: secret "image-registry-tls" not found
Sep 11 04:41:06.432209 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:06.432118    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls podName:a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:08.432100057 +0000 UTC m=+62.723574486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls") pod "image-registry-85fd7c84f5-2d5rw" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4") : secret "image-registry-tls" not found
Sep 11 04:41:07.565451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:07.565416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp" event={"ID":"276dc052-1ce1-4884-ae66-788ebe3d5fd8","Type":"ContainerStarted","Data":"9e8fb019b963d75ad3c6d9666fd1256f0c8988f274e5a24b6f14ffcb5ba9b40f"}
Sep 11 04:41:07.581124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:07.581020    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-kb9gp" podStartSLOduration=2.241646226 podStartE2EDuration="3.581001867s" totalImagesPullingTime="1.339355641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:07.579842141 +0000 UTC m=+61.871316578" watchObservedRunningTime="2026-09-11 04:41:07.581001867 +0000 UTC m=+61.872476304"
Sep 11 04:41:08.350372 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.350341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:08.350501 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:08.350481    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:08.350564 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:08.350551    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:12.350532471 +0000 UTC m=+66.642006905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:08.451329 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.451228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:08.451396 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:08.451379    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:41:08.451916 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:08.451400    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd7c84f5-2d5rw: secret "image-registry-tls" not found
Sep 11 04:41:08.451916 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:08.451453    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls podName:a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:12.451437677 +0000 UTC m=+66.742912096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls") pod "image-registry-85fd7c84f5-2d5rw" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4") : secret "image-registry-tls" not found
Sep 11 04:41:08.569565 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.569533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jm555" event={"ID":"364e817b-78a5-4e76-8fb8-6ac3644bbe8b","Type":"ContainerStarted","Data":"b4bdc9261c65724387f9caa5dba1dbe689cd705ec8658e57dac0f1bf23096830"}
Sep 11 04:41:08.571205 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.571149    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk" event={"ID":"a7a4371a-2787-4d9f-870e-e7c137a75ed7","Type":"ContainerStarted","Data":"6558a1427a264922d747dd8ae2a47267357bc6c4460bb9281f6c4282a0096437"}
Sep 11 04:41:08.572851 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.572833    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/0.log"
Sep 11 04:41:08.572947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.572869    2581 generic.go:356] "Generic (PLEG): container finished" podID="c127bc70-006d-4bf8-adab-0d937a3b5102" containerID="94607e9490573cb53bc33401b26abd8638bee49197b4687ec4cfcc41f3d6c32a" exitCode=255
Sep 11 04:41:08.572947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.572938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" event={"ID":"c127bc70-006d-4bf8-adab-0d937a3b5102","Type":"ContainerDied","Data":"94607e9490573cb53bc33401b26abd8638bee49197b4687ec4cfcc41f3d6c32a"}
Sep 11 04:41:08.573222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.573207    2581 scope.go:117] "RemoveContainer" containerID="94607e9490573cb53bc33401b26abd8638bee49197b4687ec4cfcc41f3d6c32a"
Sep 11 04:41:08.585362 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.585328    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-jm555" podStartSLOduration=1.52937233 podStartE2EDuration="4.585314017s" totalImagesPullingTime="3.055941687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:08.584122508 +0000 UTC m=+62.875596946" watchObservedRunningTime="2026-09-11 04:41:08.585314017 +0000 UTC m=+62.876788455"
Sep 11 04:41:08.598741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:08.598703    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk" podStartSLOduration=1.416544733 podStartE2EDuration="4.598689405s" totalImagesPullingTime="3.182144672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:08.598617764 +0000 UTC m=+62.890092202" watchObservedRunningTime="2026-09-11 04:41:08.598689405 +0000 UTC m=+62.890163841"
Sep 11 04:41:09.578156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578125    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/1.log"
Sep 11 04:41:09.578592 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/0.log"
Sep 11 04:41:09.578647 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578614    2581 generic.go:356] "Generic (PLEG): container finished" podID="c127bc70-006d-4bf8-adab-0d937a3b5102" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9" exitCode=255
Sep 11 04:41:09.578715 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" event={"ID":"c127bc70-006d-4bf8-adab-0d937a3b5102","Type":"ContainerDied","Data":"fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"}
Sep 11 04:41:09.578786 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578773    2581 scope.go:117] "RemoveContainer" containerID="94607e9490573cb53bc33401b26abd8638bee49197b4687ec4cfcc41f3d6c32a"
Sep 11 04:41:09.579008 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:09.578988    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:09.579183 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:09.579161    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:10.036773 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.036742    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7qwrd"]
Sep 11 04:41:10.039770 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.039748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.042073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.042050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 04:41:10.042685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.042661    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 04:41:10.042761 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.042721    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h277b\""
Sep 11 04:41:10.054666 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.054646    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7qwrd"]
Sep 11 04:41:10.065916 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.065890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.066002 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.065928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.066002 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.065967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be40e39b-4930-4de4-a1f9-43da0959c7b7-data-volume\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.066082 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.066027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7pcr\" (UniqueName: \"kubernetes.io/projected/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-api-access-q7pcr\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.066117 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.066097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be40e39b-4930-4de4-a1f9-43da0959c7b7-crio-socket\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166506 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7pcr\" (UniqueName: \"kubernetes.io/projected/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-api-access-q7pcr\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166631 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be40e39b-4930-4de4-a1f9-43da0959c7b7-crio-socket\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166631 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166631 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166678    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be40e39b-4930-4de4-a1f9-43da0959c7b7-crio-socket\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.166700    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.166717    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be40e39b-4930-4de4-a1f9-43da0959c7b7-data-volume\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.166762    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:10.666745444 +0000 UTC m=+64.958219860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.166791 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.166778    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:10.666770519 +0000 UTC m=+64.958244933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.166998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.166980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be40e39b-4930-4de4-a1f9-43da0959c7b7-data-volume\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.176254 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.176231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7pcr\" (UniqueName: \"kubernetes.io/projected/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-api-access-q7pcr\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.583107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.583079    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/1.log"
Sep 11 04:41:10.583451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.583374    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:10.583538 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.583520    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:10.671149 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.671118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.671308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:10.671163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:10.671308 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.671239    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.671308 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.671290    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.671444 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.671320    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.671300279 +0000 UTC m=+65.962774716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.671444 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:10.671341    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.67132712 +0000 UTC m=+65.962801550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:11.680490 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.680459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:11.680490 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.680492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:11.680913 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.680600    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:11.680913 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.680609    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:11.680913 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.680664    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:13.680651162 +0000 UTC m=+67.972125577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:11.680913 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.680676    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:13.680670725 +0000 UTC m=+67.972145139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:11.881770 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.881741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:41:11.884320 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.884303    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 04:41:11.892133 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.892118    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 04:41:11.892183 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.892163    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs podName:66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7 nodeName:}" failed. No retries permitted until 2026-09-11 04:42:15.892150017 +0000 UTC m=+130.183624437 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs") pod "network-metrics-daemon-2hqzh" (UID: "66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7") : secret "metrics-daemon-secret" not found
Sep 11 04:41:11.982643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.982620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:11.982746 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.982735    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:41:11.982787 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:11.982778    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls podName:a78ba00e-4e12-4712-9f78-408d47756b56 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:43.982767072 +0000 UTC m=+98.274241487 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls") pod "dns-default-42vp6" (UID: "a78ba00e-4e12-4712-9f78-408d47756b56") : secret "dns-default-metrics-tls" not found
Sep 11 04:41:11.997876 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:11.997854    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-md27b_dd6ab5b1-010f-470d-aed1-b35adf58e938/dns-node-resolver/0.log"
Sep 11 04:41:12.083570 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.083545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:41:12.083675 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.083619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:41:12.086489 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.086472    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 04:41:12.086524 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.086479    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 04:41:12.096454 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.096435    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 04:41:12.096752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.096735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/821978bb-a169-482b-bfde-a70dcbc646d4-original-pull-secret\") pod \"global-pull-secret-syncer-72h62\" (UID: \"821978bb-a169-482b-bfde-a70dcbc646d4\") " pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:41:12.107470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.107448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcfgd\" (UniqueName: \"kubernetes.io/projected/bca39b9b-29ed-46e8-8396-bec1d86aae18-kube-api-access-rcfgd\") pod \"network-check-target-psmdc\" (UID: \"bca39b9b-29ed-46e8-8396-bec1d86aae18\") " pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:41:12.386081 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.385999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:12.386222 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:12.386106    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:12.386222 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:12.386159    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:20.386144847 +0000 UTC m=+74.677619268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:12.395712 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.395693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72h62"
Sep 11 04:41:12.408369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.408352    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kl86\""
Sep 11 04:41:12.417013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.416998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:41:12.487101 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.486878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:12.487501 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:12.487315    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:41:12.487501 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:12.487400    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85fd7c84f5-2d5rw: secret "image-registry-tls" not found
Sep 11 04:41:12.487501 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:12.487484    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls podName:a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:20.48746048 +0000 UTC m=+74.778934908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls") pod "image-registry-85fd7c84f5-2d5rw" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4") : secret "image-registry-tls" not found
Sep 11 04:41:12.681177 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.681155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72h62"]
Sep 11 04:41:12.683595 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:12.683562    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821978bb_a169_482b_bfde_a70dcbc646d4.slice/crio-60010d012699442a7c9abcf736e4055321178bb7c2eed6cbe24e08a93619ae49 WatchSource:0}: Error finding container 60010d012699442a7c9abcf736e4055321178bb7c2eed6cbe24e08a93619ae49: Status 404 returned error can't find the container with id 60010d012699442a7c9abcf736e4055321178bb7c2eed6cbe24e08a93619ae49
Sep 11 04:41:12.702347 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.702326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-psmdc"]
Sep 11 04:41:12.704780 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:12.704759    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca39b9b_29ed_46e8_8396_bec1d86aae18.slice/crio-29c25a56c6c891609c5a947adcfc76489c72ef8c28fb431d764a98f7ee2219fc WatchSource:0}: Error finding container 29c25a56c6c891609c5a947adcfc76489c72ef8c28fb431d764a98f7ee2219fc: Status 404 returned error can't find the container with id 29c25a56c6c891609c5a947adcfc76489c72ef8c28fb431d764a98f7ee2219fc
Sep 11 04:41:12.797632 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:12.797604    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2wtlv_ddecf3f4-042d-4f9f-b94e-c9deff6d68b0/node-ca/0.log"
Sep 11 04:41:13.593020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:13.592954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72h62" event={"ID":"821978bb-a169-482b-bfde-a70dcbc646d4","Type":"ContainerStarted","Data":"60010d012699442a7c9abcf736e4055321178bb7c2eed6cbe24e08a93619ae49"}
Sep 11 04:41:13.594961 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:13.594936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-psmdc" event={"ID":"bca39b9b-29ed-46e8-8396-bec1d86aae18","Type":"ContainerStarted","Data":"29c25a56c6c891609c5a947adcfc76489c72ef8c28fb431d764a98f7ee2219fc"}
Sep 11 04:41:13.695095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:13.695068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:13.695477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:13.695108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:13.695477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:13.695202    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:13.695477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:13.695209    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:13.695477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:13.695282    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:17.695263469 +0000 UTC m=+71.986737900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:13.695477 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:13.695299    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:17.695290586 +0000 UTC m=+71.986765008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:13.997737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:13.997704    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xtll9_6fa39515-1242-4d0e-a11d-4218cfa55fb9/serve-healthcheck-canary/0.log"
Sep 11 04:41:15.040240 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:15.040205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:15.040716 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:15.040662    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:15.040857 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:15.040826    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:15.041234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:15.041173    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:15.601216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:15.601189    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:15.601406 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:15.601382    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:17.607519 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.607425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-psmdc" event={"ID":"bca39b9b-29ed-46e8-8396-bec1d86aae18","Type":"ContainerStarted","Data":"7297b4f69ea0fe2041ab381d092aad4ec23c79930108f7e27f1a161a7af82a19"}
Sep 11 04:41:17.607906 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.607524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:41:17.608943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.608904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72h62" event={"ID":"821978bb-a169-482b-bfde-a70dcbc646d4","Type":"ContainerStarted","Data":"c6e5ef2923e3c2299b934fc291651b7bf263876e6d87f70103fc474fefcd083d"}
Sep 11 04:41:17.623936 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.623888    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-psmdc" podStartSLOduration=67.052480078 podStartE2EDuration="1m11.623874306s" totalImagesPullingTime="4.571394228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:17.62257366 +0000 UTC m=+71.914048096" watchObservedRunningTime="2026-09-11 04:41:17.623874306 +0000 UTC m=+71.915348742"
Sep 11 04:41:17.636175 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.636131    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-72h62" podStartSLOduration=67.040957686 podStartE2EDuration="1m11.636117267s" totalImagesPullingTime="4.595159581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:17.63571119 +0000 UTC m=+71.927185625" watchObservedRunningTime="2026-09-11 04:41:17.636117267 +0000 UTC m=+71.927591743"
Sep 11 04:41:17.725171 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.725138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:17.725275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:17.725185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:17.725275 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:17.725262    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:17.725379 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:17.725316    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:25.725300385 +0000 UTC m=+80.016774800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:17.725379 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:17.725353    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:17.725468 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:17.725395    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls podName:be40e39b-4930-4de4-a1f9-43da0959c7b7 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:25.725383401 +0000 UTC m=+80.016857816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qwrd" (UID: "be40e39b-4930-4de4-a1f9-43da0959c7b7") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:20.446077 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.446033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:20.446480 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:20.446186    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:20.446480 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:20.446253    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls podName:c605a2a6-fbf1-47cf-a5d9-624aedca0a01 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:36.446236728 +0000 UTC m=+90.737711148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hvtsp" (UID: "c605a2a6-fbf1-47cf-a5d9-624aedca0a01") : secret "cluster-monitoring-operator-tls" not found
Sep 11 04:41:20.546397 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.546362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:20.548845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.548828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"image-registry-85fd7c84f5-2d5rw\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") " pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:20.653521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.653499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkc4\""
Sep 11 04:41:20.662284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.662264    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:20.948741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:20.948704    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:41:20.953196 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:20.953171    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f85213_f16b_4bb1_93cc_0a1f9c8fe6d4.slice/crio-254d6ea1e51fac304408f9b88925973d1956b56f412599df0dd4364a4500e0e0 WatchSource:0}: Error finding container 254d6ea1e51fac304408f9b88925973d1956b56f412599df0dd4364a4500e0e0: Status 404 returned error can't find the container with id 254d6ea1e51fac304408f9b88925973d1956b56f412599df0dd4364a4500e0e0
Sep 11 04:41:21.624724 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:21.624689    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" event={"ID":"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4","Type":"ContainerStarted","Data":"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"}
Sep 11 04:41:21.625137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:21.624746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" event={"ID":"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4","Type":"ContainerStarted","Data":"254d6ea1e51fac304408f9b88925973d1956b56f412599df0dd4364a4500e0e0"}
Sep 11 04:41:21.625137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:21.624827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:21.643359 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:21.643309    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" podStartSLOduration=17.643296963 podStartE2EDuration="17.643296963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:21.641982743 +0000 UTC m=+75.933457179" watchObservedRunningTime="2026-09-11 04:41:21.643296963 +0000 UTC m=+75.934771400"
Sep 11 04:41:25.785137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:25.785102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:25.785137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:25.785136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:25.785654 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:25.785637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be40e39b-4930-4de4-a1f9-43da0959c7b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:25.787481 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:25.787466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be40e39b-4930-4de4-a1f9-43da0959c7b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qwrd\" (UID: \"be40e39b-4930-4de4-a1f9-43da0959c7b7\") " pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:25.948706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:25.948663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7qwrd"
Sep 11 04:41:26.215698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:26.215562    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7qwrd"]
Sep 11 04:41:26.217977 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:26.217956    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe40e39b_4930_4de4_a1f9_43da0959c7b7.slice/crio-5c78ab5fced9f7ca4a829e548019a3f419cbcc9ce51eff86f13b96cf15f21202 WatchSource:0}: Error finding container 5c78ab5fced9f7ca4a829e548019a3f419cbcc9ce51eff86f13b96cf15f21202: Status 404 returned error can't find the container with id 5c78ab5fced9f7ca4a829e548019a3f419cbcc9ce51eff86f13b96cf15f21202
Sep 11 04:41:26.642312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:26.642234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qwrd" event={"ID":"be40e39b-4930-4de4-a1f9-43da0959c7b7","Type":"ContainerStarted","Data":"4cb4ca1c9e0ba6ed93431aa188e7d7d24d6c3af23cdf9ef0f708184ff5dbb310"}
Sep 11 04:41:26.642312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:26.642291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qwrd" event={"ID":"be40e39b-4930-4de4-a1f9-43da0959c7b7","Type":"ContainerStarted","Data":"5c78ab5fced9f7ca4a829e548019a3f419cbcc9ce51eff86f13b96cf15f21202"}
Sep 11 04:41:27.647485 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:27.647445    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qwrd" event={"ID":"be40e39b-4930-4de4-a1f9-43da0959c7b7","Type":"ContainerStarted","Data":"012abfc8e3693eadc771fc6a84ba9088abb96cff9cec64679910ee471380085c"}
Sep 11 04:41:28.652638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:28.652528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qwrd" event={"ID":"be40e39b-4930-4de4-a1f9-43da0959c7b7","Type":"ContainerStarted","Data":"5cd291d9c83031b8de3a9fc52d0cb8f2f34d93656c18eddb577ecfd179ed7be6"}
Sep 11 04:41:28.670095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:28.670047    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7qwrd" podStartSLOduration=16.66750925 podStartE2EDuration="18.670030637s" totalImagesPullingTime="2.002521387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:41:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:28.668856626 +0000 UTC m=+82.960331065" watchObservedRunningTime="2026-09-11 04:41:28.670030637 +0000 UTC m=+82.961505071"
Sep 11 04:41:30.187116 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.187081    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:30.659952 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.659920    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:41:30.660337 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.660323    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/1.log"
Sep 11 04:41:30.660399 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.660356    2581 generic.go:356] "Generic (PLEG): container finished" podID="c127bc70-006d-4bf8-adab-0d937a3b5102" containerID="da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704" exitCode=255
Sep 11 04:41:30.660455 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.660433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" event={"ID":"c127bc70-006d-4bf8-adab-0d937a3b5102","Type":"ContainerDied","Data":"da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704"}
Sep 11 04:41:30.660526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.660516    2581 scope.go:117] "RemoveContainer" containerID="fbc2eb5e0181421ec7ec7176537c0e56567f7430612f7336676be699d02712b9"
Sep 11 04:41:30.660806 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:30.660791    2581 scope.go:117] "RemoveContainer" containerID="da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704"
Sep 11 04:41:30.661018 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:30.660994    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:31.664646 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:31.664620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:41:33.198784 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.198755    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"]
Sep 11 04:41:33.200980 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.200960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.203146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.203122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 04:41:33.203146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.203141    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 04:41:33.203625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.203606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 04:41:33.204198 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.204163    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 04:41:33.204283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.204216    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 04:41:33.204283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.204241    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 04:41:33.204425 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.204406    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 04:41:33.207810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.207511    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:41:33.218995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.218973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"]
Sep 11 04:41:33.242203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.242319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.242319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.242319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.242438 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4ec864-7297-4348-a3fd-43b0fcec18e8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.242438 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.242407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwlv\" (UniqueName: \"kubernetes.io/projected/3d4ec864-7297-4348-a3fd-43b0fcec18e8-kube-api-access-fkwlv\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343520 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343630 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4ec864-7297-4348-a3fd-43b0fcec18e8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343630 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwlv\" (UniqueName: \"kubernetes.io/projected/3d4ec864-7297-4348-a3fd-43b0fcec18e8-kube-api-access-fkwlv\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343743 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343796 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.343796 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.343772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.344254 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.344227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4ec864-7297-4348-a3fd-43b0fcec18e8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.346067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.346038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.346209 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.346189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.346209 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.346201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-ca\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.346296 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.346200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d4ec864-7297-4348-a3fd-43b0fcec18e8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.355643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.355620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwlv\" (UniqueName: \"kubernetes.io/projected/3d4ec864-7297-4348-a3fd-43b0fcec18e8-kube-api-access-fkwlv\") pod \"cluster-proxy-proxy-agent-fdd754f54-hv9q2\" (UID: \"3d4ec864-7297-4348-a3fd-43b0fcec18e8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.520360 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.520272    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"
Sep 11 04:41:33.802714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:33.802538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2"]
Sep 11 04:41:33.805029 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:33.805008    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d4ec864_7297_4348_a3fd_43b0fcec18e8.slice/crio-932ae2e33ed612c5f129dddde3700390da41eb82cba6690f3895b159ac22b8b7 WatchSource:0}: Error finding container 932ae2e33ed612c5f129dddde3700390da41eb82cba6690f3895b159ac22b8b7: Status 404 returned error can't find the container with id 932ae2e33ed612c5f129dddde3700390da41eb82cba6690f3895b159ac22b8b7
Sep 11 04:41:34.676732 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:34.676696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2" event={"ID":"3d4ec864-7297-4348-a3fd-43b0fcec18e8","Type":"ContainerStarted","Data":"932ae2e33ed612c5f129dddde3700390da41eb82cba6690f3895b159ac22b8b7"}
Sep 11 04:41:35.039823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:35.039786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:35.039984 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:35.039883    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:41:35.040303 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:35.040219    2581 scope.go:117] "RemoveContainer" containerID="da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704"
Sep 11 04:41:35.040446 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:35.040418    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:36.470155 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:36.470115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:36.472838 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:36.472813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c605a2a6-fbf1-47cf-a5d9-624aedca0a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hvtsp\" (UID: \"c605a2a6-fbf1-47cf-a5d9-624aedca0a01\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:36.737926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:36.737844    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6ghxt\""
Sep 11 04:41:36.746493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:36.746464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"
Sep 11 04:41:37.349213 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.349157    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp"]
Sep 11 04:41:37.351886 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:37.351864    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc605a2a6_fbf1_47cf_a5d9_624aedca0a01.slice/crio-eef3dd0c07a8cbe7d42b88bddd95e352506df3bf4ed0ad8b1e4d05770fa19527 WatchSource:0}: Error finding container eef3dd0c07a8cbe7d42b88bddd95e352506df3bf4ed0ad8b1e4d05770fa19527: Status 404 returned error can't find the container with id eef3dd0c07a8cbe7d42b88bddd95e352506df3bf4ed0ad8b1e4d05770fa19527
Sep 11 04:41:37.688199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.688117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp" event={"ID":"c605a2a6-fbf1-47cf-a5d9-624aedca0a01","Type":"ContainerStarted","Data":"eef3dd0c07a8cbe7d42b88bddd95e352506df3bf4ed0ad8b1e4d05770fa19527"}
Sep 11 04:41:37.690059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.690036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2" event={"ID":"3d4ec864-7297-4348-a3fd-43b0fcec18e8","Type":"ContainerStarted","Data":"590d7eee0292cec61292b1ecd34e952be2e5b057473ec2de85287436fdf46f00"}
Sep 11 04:41:37.690171 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.690070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2" event={"ID":"3d4ec864-7297-4348-a3fd-43b0fcec18e8","Type":"ContainerStarted","Data":"21d8f537bf7ff96a5da30578e7a0a993175014abf2c3f3848aeeaaf48efe9de3"}
Sep 11 04:41:37.690171 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.690088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2" event={"ID":"3d4ec864-7297-4348-a3fd-43b0fcec18e8","Type":"ContainerStarted","Data":"95c902d0abfc6fbaa12535137f789a485bb2cd166e6bdbc60a453ed653e4bf28"}
Sep 11 04:41:37.707206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:37.707167    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fdd754f54-hv9q2" podStartSLOduration=1.368892001 podStartE2EDuration="4.707153749s" totalImagesPullingTime="3.338261748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:37.706369303 +0000 UTC m=+91.997843739" watchObservedRunningTime="2026-09-11 04:41:37.707153749 +0000 UTC m=+91.998628185"
Sep 11 04:41:39.700455 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:39.700415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp" event={"ID":"c605a2a6-fbf1-47cf-a5d9-624aedca0a01","Type":"ContainerStarted","Data":"d7e62b663b74219da2524398fab6b062530e1d79713c77bbe2e0959e85c95105"}
Sep 11 04:41:39.716428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:39.716387    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hvtsp" podStartSLOduration=34.201325369 podStartE2EDuration="35.716373176s" totalImagesPullingTime="1.515047807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:39.71563106 +0000 UTC m=+94.007105497" watchObservedRunningTime="2026-09-11 04:41:39.716373176 +0000 UTC m=+94.007847612"
Sep 11 04:41:43.213292 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:43.213253    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:44.027652 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.027620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:44.029905 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.029886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78ba00e-4e12-4712-9f78-408d47756b56-metrics-tls\") pod \"dns-default-42vp6\" (UID: \"a78ba00e-4e12-4712-9f78-408d47756b56\") " pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:44.320233 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.320159    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v459k\""
Sep 11 04:41:44.328995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.328964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:44.602765 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.602696    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-42vp6"]
Sep 11 04:41:44.719070 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:44.719041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-42vp6" event={"ID":"a78ba00e-4e12-4712-9f78-408d47756b56","Type":"ContainerStarted","Data":"ab15eb4b78bb9c742ed617eac148c624b0d913cb6fecc3ebc08ee5a9c5dba602"}
Sep 11 04:41:46.730924 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:46.730890    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-42vp6" event={"ID":"a78ba00e-4e12-4712-9f78-408d47756b56","Type":"ContainerStarted","Data":"47f1b8194e31ec161d62323bd4d7853d94dece36875a2c54900b15d5f99a5eef"}
Sep 11 04:41:46.731377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:46.730950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-42vp6" event={"ID":"a78ba00e-4e12-4712-9f78-408d47756b56","Type":"ContainerStarted","Data":"dcee19f2de72fa2b2541b64484388f9df12aac55b1586b34fbbbcdf60a1c6195"}
Sep 11 04:41:46.731377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:46.731011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:46.746969 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:46.746921    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-42vp6" podStartSLOduration=64.940956635 podStartE2EDuration="1m6.746905827s" totalImagesPullingTime="1.805949192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:46.745986937 +0000 UTC m=+101.037461374" watchObservedRunningTime="2026-09-11 04:41:46.746905827 +0000 UTC m=+101.038380263"
Sep 11 04:41:47.187360 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:47.187294    2581 scope.go:117] "RemoveContainer" containerID="da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704"
Sep 11 04:41:47.187527 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:47.187442    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-tzsh8_openshift-console-operator(c127bc70-006d-4bf8-adab-0d937a3b5102)\"" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podUID="c127bc70-006d-4bf8-adab-0d937a3b5102"
Sep 11 04:41:48.615324 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.615293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-psmdc"
Sep 11 04:41:48.796219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.796189    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-crht5"]
Sep 11 04:41:48.819169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.819147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.821984 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.821966    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hr4hw\""
Sep 11 04:41:48.822276 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.822256    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 04:41:48.822337 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.822324    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 04:41:48.822505 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.822489    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 04:41:48.822665 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.822648    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 04:41:48.966212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-wtmp\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966326 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-metrics-client-ca\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966326 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-accelerators-collector-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966326 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966326 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-sys\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-root\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-tls\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-textfile\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:48.966482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:48.966415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvhwl\" (UniqueName: \"kubernetes.io/projected/e90d2a02-1fe3-4598-a64e-2cef26c5154f-kube-api-access-vvhwl\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067436 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-sys\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-root\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-tls\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-textfile\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvhwl\" (UniqueName: \"kubernetes.io/projected/e90d2a02-1fe3-4598-a64e-2cef26c5154f-kube-api-access-vvhwl\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-sys\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-root\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067676    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-wtmp\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-metrics-client-ca\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.067781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-accelerators-collector-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.068073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067789    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-wtmp\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.068073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.067885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-textfile\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.068217 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.068197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-metrics-client-ca\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.068278 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.068238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-accelerators-collector-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.070507 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.070484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.070617 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.070551    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e90d2a02-1fe3-4598-a64e-2cef26c5154f-node-exporter-tls\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.077526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.077505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvhwl\" (UniqueName: \"kubernetes.io/projected/e90d2a02-1fe3-4598-a64e-2cef26c5154f-kube-api-access-vvhwl\") pod \"node-exporter-crht5\" (UID: \"e90d2a02-1fe3-4598-a64e-2cef26c5154f\") " pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.128230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.128200    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-crht5"
Sep 11 04:41:49.160094 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:41:49.160071    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode90d2a02_1fe3_4598_a64e_2cef26c5154f.slice/crio-849bdb990ed34f42140d1dfa42ded32e04ccdc2bb649bc450ebe48890661d156 WatchSource:0}: Error finding container 849bdb990ed34f42140d1dfa42ded32e04ccdc2bb649bc450ebe48890661d156: Status 404 returned error can't find the container with id 849bdb990ed34f42140d1dfa42ded32e04ccdc2bb649bc450ebe48890661d156
Sep 11 04:41:49.744400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:49.744366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-crht5" event={"ID":"e90d2a02-1fe3-4598-a64e-2cef26c5154f","Type":"ContainerStarted","Data":"849bdb990ed34f42140d1dfa42ded32e04ccdc2bb649bc450ebe48890661d156"}
Sep 11 04:41:50.750053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:50.750020    2581 generic.go:356] "Generic (PLEG): container finished" podID="e90d2a02-1fe3-4598-a64e-2cef26c5154f" containerID="0fda5f9e8e5611e1422c68523168089d872143d15bea10329702cf458a712802" exitCode=0
Sep 11 04:41:50.750467 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:50.750109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-crht5" event={"ID":"e90d2a02-1fe3-4598-a64e-2cef26c5154f","Type":"ContainerDied","Data":"0fda5f9e8e5611e1422c68523168089d872143d15bea10329702cf458a712802"}
Sep 11 04:41:51.756071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:51.756028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-crht5" event={"ID":"e90d2a02-1fe3-4598-a64e-2cef26c5154f","Type":"ContainerStarted","Data":"97b606c4e4fcdbfbfdf25eb03c7d36f9bfe6739169c04f5e027491c1bfd0660e"}
Sep 11 04:41:51.756532 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:51.756088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-crht5" event={"ID":"e90d2a02-1fe3-4598-a64e-2cef26c5154f","Type":"ContainerStarted","Data":"ab94bc3b214a3063452977db9320daca524fb8adfaa6c765542899e3fd133033"}
Sep 11 04:41:51.775657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:51.775614    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-crht5" podStartSLOduration=2.641385293 podStartE2EDuration="3.775599376s" totalImagesPullingTime="1.06691532s" totalInitContainerRuntime="67.298763ms" isStatefulPod=true podCreationTimestamp="2026-09-11 04:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:51.774043332 +0000 UTC m=+106.065517770" watchObservedRunningTime="2026-09-11 04:41:51.775599376 +0000 UTC m=+106.067073817"
Sep 11 04:41:56.738791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:56.738762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-42vp6"
Sep 11 04:41:58.228283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.228241    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" podUID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" containerName="registry" containerID="cri-o://7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca" gracePeriod=30
Sep 11 04:41:58.466616 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.466572    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:58.638655 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638655 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638628    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638655 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638715    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638758    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.638886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.638781    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jswm8\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8\") pod \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\" (UID: \"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4\") "
Sep 11 04:41:58.639125 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.639095    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:41:58.639199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.639167    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:41:58.641367 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.641337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:41:58.641594 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.641537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:41:58.641701 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.641573    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:41:58.641701 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.641649    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:41:58.641701 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.641660    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8" (OuterVolumeSpecName: "kube-api-access-jswm8") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "kube-api-access-jswm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:41:58.647702 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.647681    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" (UID: "a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:41:58.739587 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739564    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-tls\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739604    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jswm8\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-kube-api-access-jswm8\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739615    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-registry-certificates\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739624    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-ca-trust-extracted\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739634    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-image-registry-private-configuration\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739643    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-trusted-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739652    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-installation-pull-secrets\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.739670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.739660    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4-bound-sa-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:41:58.781934 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.781911    2581 generic.go:356] "Generic (PLEG): container finished" podID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" containerID="7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca" exitCode=0
Sep 11 04:41:58.782007 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.781956    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw"
Sep 11 04:41:58.782053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.781940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" event={"ID":"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4","Type":"ContainerDied","Data":"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"}
Sep 11 04:41:58.782087 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.782065    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85fd7c84f5-2d5rw" event={"ID":"a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4","Type":"ContainerDied","Data":"254d6ea1e51fac304408f9b88925973d1956b56f412599df0dd4364a4500e0e0"}
Sep 11 04:41:58.782087 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.782084    2581 scope.go:117] "RemoveContainer" containerID="7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"
Sep 11 04:41:58.790020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.790005    2581 scope.go:117] "RemoveContainer" containerID="7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"
Sep 11 04:41:58.790271 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:41:58.790255    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca\": container with ID starting with 7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca not found: ID does not exist" containerID="7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"
Sep 11 04:41:58.790319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.790277    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca"} err="failed to get container status \"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca\": rpc error: code = NotFound desc = could not find container \"7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca\": container with ID starting with 7a07e5f912a33871cecedf96614d437f283a9ea44a135fb7d436431b6ff63cca not found: ID does not exist"
Sep 11 04:41:58.801725 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.801705    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:41:58.809034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:41:58.809015    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85fd7c84f5-2d5rw"]
Sep 11 04:42:00.190859 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:00.190821    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" path="/var/lib/kubelet/pods/a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4/volumes"
Sep 11 04:42:02.187413 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.187379    2581 scope.go:117] "RemoveContainer" containerID="da726e4599236570e0f438e700146b6226b251b3400381a3447f0cead068b704"
Sep 11 04:42:02.802801 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.802775    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:42:02.802958 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.802881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" event={"ID":"c127bc70-006d-4bf8-adab-0d937a3b5102","Type":"ContainerStarted","Data":"19d33d1ce745904e3394730ffd9f2621c19b01a588840fd4f18325b3b3ab7d44"}
Sep 11 04:42:02.803179 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.803153    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:42:02.807760 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.807742    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8"
Sep 11 04:42:02.824829 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:02.824791    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-tzsh8" podStartSLOduration=55.813279895 podStartE2EDuration="58.824779009s" totalImagesPullingTime="3.011499114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:42:02.823690428 +0000 UTC m=+117.115164867" watchObservedRunningTime="2026-09-11 04:42:02.824779009 +0000 UTC m=+117.116253491"
Sep 11 04:42:14.844266 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:14.844185    2581 generic.go:356] "Generic (PLEG): container finished" podID="364e817b-78a5-4e76-8fb8-6ac3644bbe8b" containerID="b4bdc9261c65724387f9caa5dba1dbe689cd705ec8658e57dac0f1bf23096830" exitCode=0
Sep 11 04:42:14.844671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:14.844258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jm555" event={"ID":"364e817b-78a5-4e76-8fb8-6ac3644bbe8b","Type":"ContainerDied","Data":"b4bdc9261c65724387f9caa5dba1dbe689cd705ec8658e57dac0f1bf23096830"}
Sep 11 04:42:14.844671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:14.844657    2581 scope.go:117] "RemoveContainer" containerID="b4bdc9261c65724387f9caa5dba1dbe689cd705ec8658e57dac0f1bf23096830"
Sep 11 04:42:15.132543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:15.132463    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xtll9_6fa39515-1242-4d0e-a11d-4218cfa55fb9/serve-healthcheck-canary/0.log"
Sep 11 04:42:15.849212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:15.849176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jm555" event={"ID":"364e817b-78a5-4e76-8fb8-6ac3644bbe8b","Type":"ContainerStarted","Data":"8630ee9a1612347a5eb00f725219b8aed314bd39d440fc18c4560da62046435a"}
Sep 11 04:42:15.966638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:15.966603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:42:15.968994 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:15.968973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7-metrics-certs\") pod \"network-metrics-daemon-2hqzh\" (UID: \"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7\") " pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:42:16.004230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.004197    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rs4qk\""
Sep 11 04:42:16.012931 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.012902    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hqzh"
Sep 11 04:42:16.282352 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.282324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hqzh"]
Sep 11 04:42:16.284980 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:42:16.284948    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f0bfa2_d2e2_41a5_a273_2d56ec18f9e7.slice/crio-36b5c483c26b3f19ea78d3f2aba314626e22295e6592c024b582b90e85497b20 WatchSource:0}: Error finding container 36b5c483c26b3f19ea78d3f2aba314626e22295e6592c024b582b90e85497b20: Status 404 returned error can't find the container with id 36b5c483c26b3f19ea78d3f2aba314626e22295e6592c024b582b90e85497b20
Sep 11 04:42:16.667202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.667122    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:42:16.667444 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.667429    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" containerName="registry"
Sep 11 04:42:16.667517 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.667446    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" containerName="registry"
Sep 11 04:42:16.667517 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.667502    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0f85213-f16b-4bb1-93cc-0a1f9c8fe6d4" containerName="registry"
Sep 11 04:42:16.669297 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.669281    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.671426 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.671405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 04:42:16.671695 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.671675    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 04:42:16.671923 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.671907    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5f9r2\""
Sep 11 04:42:16.672531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.672338    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 04:42:16.672531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.672346    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 04:42:16.672531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.672403    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 04:42:16.672531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.672445    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 04:42:16.672810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.672621    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 04:42:16.678168 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.678137    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 04:42:16.679111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.679061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:42:16.772095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772262 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.772526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.772378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjd4x\" (UniqueName: \"kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.854643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.854603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hqzh" event={"ID":"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7","Type":"ContainerStarted","Data":"36b5c483c26b3f19ea78d3f2aba314626e22295e6592c024b582b90e85497b20"}
Sep 11 04:42:16.873097 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.873714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.873264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjd4x\" (UniqueName: \"kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.874193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.874135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.874303 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.874197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.874849 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.874825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.874957 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.874869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.876255 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.876228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.876346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.876270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.881309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.881287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjd4x\" (UniqueName: \"kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x\") pod \"console-8486c4fc64-65flw\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") " pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:16.982249 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:16.982203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:17.417695 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:17.417667    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:42:17.420240 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:42:17.420216    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a33f772_a744_46d3_88b1_6563ea20df96.slice/crio-4c4caaf22c0ca25c07da2667ae33e72b669af9df9224d84ed20ec6d7dfcd42f2 WatchSource:0}: Error finding container 4c4caaf22c0ca25c07da2667ae33e72b669af9df9224d84ed20ec6d7dfcd42f2: Status 404 returned error can't find the container with id 4c4caaf22c0ca25c07da2667ae33e72b669af9df9224d84ed20ec6d7dfcd42f2
Sep 11 04:42:17.859280 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:17.859249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8486c4fc64-65flw" event={"ID":"3a33f772-a744-46d3-88b1-6563ea20df96","Type":"ContainerStarted","Data":"4c4caaf22c0ca25c07da2667ae33e72b669af9df9224d84ed20ec6d7dfcd42f2"}
Sep 11 04:42:17.860994 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:17.860969    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hqzh" event={"ID":"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7","Type":"ContainerStarted","Data":"874a257927e82d0defcd529e7df3922f32f86f5bd78d8cd4fbf9dba6f934a13a"}
Sep 11 04:42:17.861104 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:17.861011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hqzh" event={"ID":"66f0bfa2-d2e2-41a5-a273-2d56ec18f9e7","Type":"ContainerStarted","Data":"91fc4c0ee6fbce71eadfd7a0cf3d3df5564836d418f4b0eedab4d9eb96e14bac"}
Sep 11 04:42:17.876903 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:17.876865    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2hqzh" podStartSLOduration=130.991253141 podStartE2EDuration="2m11.876851678s" totalImagesPullingTime="885.598537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:42:17.875707865 +0000 UTC m=+132.167182304" watchObservedRunningTime="2026-09-11 04:42:17.876851678 +0000 UTC m=+132.168326115"
Sep 11 04:42:20.874059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:20.874025    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7a4371a-2787-4d9f-870e-e7c137a75ed7" containerID="6558a1427a264922d747dd8ae2a47267357bc6c4460bb9281f6c4282a0096437" exitCode=0
Sep 11 04:42:20.874490 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:20.874107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk" event={"ID":"a7a4371a-2787-4d9f-870e-e7c137a75ed7","Type":"ContainerDied","Data":"6558a1427a264922d747dd8ae2a47267357bc6c4460bb9281f6c4282a0096437"}
Sep 11 04:42:20.874490 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:20.874439    2581 scope.go:117] "RemoveContainer" containerID="6558a1427a264922d747dd8ae2a47267357bc6c4460bb9281f6c4282a0096437"
Sep 11 04:42:20.875834 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:20.875727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8486c4fc64-65flw" event={"ID":"3a33f772-a744-46d3-88b1-6563ea20df96","Type":"ContainerStarted","Data":"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"}
Sep 11 04:42:20.905479 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:20.904991    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8486c4fc64-65flw" podStartSLOduration=2.16833376 podStartE2EDuration="4.904972898s" totalImagesPullingTime="2.736639138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:42:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:42:20.904320433 +0000 UTC m=+135.195794871" watchObservedRunningTime="2026-09-11 04:42:20.904972898 +0000 UTC m=+135.196447350"
Sep 11 04:42:21.885086 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:21.885048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7mqwk" event={"ID":"a7a4371a-2787-4d9f-870e-e7c137a75ed7","Type":"ContainerStarted","Data":"8b2b98dc06a99181bc853ddadaaba424652ac810a70678b26cb743eab616ac21"}
Sep 11 04:42:26.982871 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:26.982833    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:26.983235 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:26.982946    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:26.988091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:26.988065    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:42:27.914547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:42:27.914521    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:43:45.669409 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.669370    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:43:45.672408 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.672385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.690457 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.690431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:43:45.800017 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.799976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf7mz\" (UniqueName: \"kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.800369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.800258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901356 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901356 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.901569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.901477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7mz\" (UniqueName: \"kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.902267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.902235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.902267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.902246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.902427 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.902316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.902427 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.902316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.904004 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.903986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.904122 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.904103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.908909 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.908887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7mz\" (UniqueName: \"kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz\") pod \"console-5544b7bd74-k9lzq\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") " pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:45.981896 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:45.981862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:46.254810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:46.254735    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:43:46.259018 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:43:46.258990    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b757413_0a92_442d_8b0a_44735da25aa7.slice/crio-3709a9724512c3a9e700b13d400a370aa64dc1f9dec37b531e768a9d8520d6c0 WatchSource:0}: Error finding container 3709a9724512c3a9e700b13d400a370aa64dc1f9dec37b531e768a9d8520d6c0: Status 404 returned error can't find the container with id 3709a9724512c3a9e700b13d400a370aa64dc1f9dec37b531e768a9d8520d6c0
Sep 11 04:43:47.190748 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:47.190717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5544b7bd74-k9lzq" event={"ID":"3b757413-0a92-442d-8b0a-44735da25aa7","Type":"ContainerStarted","Data":"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"}
Sep 11 04:43:47.191121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:47.190763    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5544b7bd74-k9lzq" event={"ID":"3b757413-0a92-442d-8b0a-44735da25aa7","Type":"ContainerStarted","Data":"3709a9724512c3a9e700b13d400a370aa64dc1f9dec37b531e768a9d8520d6c0"}
Sep 11 04:43:47.208317 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:47.208271    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5544b7bd74-k9lzq" podStartSLOduration=2.208255901 podStartE2EDuration="2.208255901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:43:47.207196766 +0000 UTC m=+221.498671203" watchObservedRunningTime="2026-09-11 04:43:47.208255901 +0000 UTC m=+221.499730391"
Sep 11 04:43:55.982559 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:55.982531    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:55.983001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:55.982643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:55.987200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:55.987178    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:56.222667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:56.222642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:43:56.264487 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:43:56.264406    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:44:21.284959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.284915    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8486c4fc64-65flw" podUID="3a33f772-a744-46d3-88b1-6563ea20df96" containerName="console" containerID="cri-o://5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22" gracePeriod=15
Sep 11 04:44:21.527456 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.527433    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:44:21.549176 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549113    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549176 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549334 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549188    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjd4x\" (UniqueName: \"kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549398 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert\") pod \"3a33f772-a744-46d3-88b1-6563ea20df96\" (UID: \"3a33f772-a744-46d3-88b1-6563ea20df96\") "
Sep 11 04:44:21.549643 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:44:21.549800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:44:21.549884 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549798    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config" (OuterVolumeSpecName: "console-config") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:44:21.549884 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549866    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:44:21.549982 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549884    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.549982 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549902    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-console-config\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.549982 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.549918    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-trusted-ca-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.551986 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.551926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x" (OuterVolumeSpecName: "kube-api-access-pjd4x") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "kube-api-access-pjd4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:44:21.552132 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.552111    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:44:21.552201 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.552182    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a33f772-a744-46d3-88b1-6563ea20df96" (UID: "3a33f772-a744-46d3-88b1-6563ea20df96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:44:21.650878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.650839    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-serving-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.650878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.650872    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a33f772-a744-46d3-88b1-6563ea20df96-console-oauth-config\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.650878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.650884    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjd4x\" (UniqueName: \"kubernetes.io/projected/3a33f772-a744-46d3-88b1-6563ea20df96-kube-api-access-pjd4x\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:21.651103 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:21.650897    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a33f772-a744-46d3-88b1-6563ea20df96-oauth-serving-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:22.044542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.044507    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"]
Sep 11 04:44:22.044785 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.044773    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a33f772-a744-46d3-88b1-6563ea20df96" containerName="console"
Sep 11 04:44:22.044831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.044787    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a33f772-a744-46d3-88b1-6563ea20df96" containerName="console"
Sep 11 04:44:22.044864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.044841    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a33f772-a744-46d3-88b1-6563ea20df96" containerName="console"
Sep 11 04:44:22.048059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.048044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.050443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.050425    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 04:44:22.050857 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.050844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 04:44:22.050987 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.050972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzm5d\""
Sep 11 04:44:22.055952 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.055933    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"]
Sep 11 04:44:22.155219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.155188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjfqg\" (UniqueName: \"kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.155219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.155225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.155433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.155250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.256111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.256071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjfqg\" (UniqueName: \"kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.256288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.256115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.256288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.256156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.256565 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.256545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.256686 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.256563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.264405 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.264382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjfqg\" (UniqueName: \"kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.305708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.305630    2581 generic.go:356] "Generic (PLEG): container finished" podID="3a33f772-a744-46d3-88b1-6563ea20df96" containerID="5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22" exitCode=0
Sep 11 04:44:22.305708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.305683    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8486c4fc64-65flw"
Sep 11 04:44:22.306100 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.305711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8486c4fc64-65flw" event={"ID":"3a33f772-a744-46d3-88b1-6563ea20df96","Type":"ContainerDied","Data":"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"}
Sep 11 04:44:22.306100 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.305767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8486c4fc64-65flw" event={"ID":"3a33f772-a744-46d3-88b1-6563ea20df96","Type":"ContainerDied","Data":"4c4caaf22c0ca25c07da2667ae33e72b669af9df9224d84ed20ec6d7dfcd42f2"}
Sep 11 04:44:22.306100 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.305787    2581 scope.go:117] "RemoveContainer" containerID="5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"
Sep 11 04:44:22.313652 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.313637    2581 scope.go:117] "RemoveContainer" containerID="5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"
Sep 11 04:44:22.313919 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:44:22.313901    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22\": container with ID starting with 5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22 not found: ID does not exist" containerID="5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"
Sep 11 04:44:22.313987 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.313927    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22"} err="failed to get container status \"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22\": rpc error: code = NotFound desc = could not find container \"5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22\": container with ID starting with 5891141f67a86760da7a7d8eb51103e536aea4342e9038827096583bbb82dc22 not found: ID does not exist"
Sep 11 04:44:22.320359 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.320337    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:44:22.324943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.324923    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8486c4fc64-65flw"]
Sep 11 04:44:22.357138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.357111    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:22.628347 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:22.628324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"]
Sep 11 04:44:22.630869 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:44:22.630837    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67aedb4f_9b56_4882_bc5f_4908c1a98c50.slice/crio-bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159 WatchSource:0}: Error finding container bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159: Status 404 returned error can't find the container with id bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159
Sep 11 04:44:23.310341 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:23.310310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerStarted","Data":"bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159"}
Sep 11 04:44:24.191115 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:24.191085    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a33f772-a744-46d3-88b1-6563ea20df96" path="/var/lib/kubelet/pods/3a33f772-a744-46d3-88b1-6563ea20df96/volumes"
Sep 11 04:44:29.336219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:29.336184    2581 generic.go:356] "Generic (PLEG): container finished" podID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerID="c391ce30bca97f23739bb7fb9600af0afc25caac1e5ec2124f16e2111ad7aa19" exitCode=0
Sep 11 04:44:29.336670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:29.336229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerDied","Data":"c391ce30bca97f23739bb7fb9600af0afc25caac1e5ec2124f16e2111ad7aa19"}
Sep 11 04:44:33.352515 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:33.352478    2581 generic.go:356] "Generic (PLEG): container finished" podID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerID="d786752bad658273e27ba291ec7e8a1d3e5b61f81c61a4c6dac4d5819edf32f0" exitCode=0
Sep 11 04:44:33.352905 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:33.352537    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerDied","Data":"d786752bad658273e27ba291ec7e8a1d3e5b61f81c61a4c6dac4d5819edf32f0"}
Sep 11 04:44:40.383384 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:40.383353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerStarted","Data":"e5be6121d8f4c94f8d903919ec701b151040829d75fa0df410754894f515f972"}
Sep 11 04:44:40.401270 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:40.401210    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" podStartSLOduration=2.132186991 podStartE2EDuration="18.401193351s" totalImagesPullingTime="16.26900636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:44:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:44:40.399461659 +0000 UTC m=+274.690936119" watchObservedRunningTime="2026-09-11 04:44:40.401193351 +0000 UTC m=+274.692667787"
Sep 11 04:44:41.389179 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:41.389146    2581 generic.go:356] "Generic (PLEG): container finished" podID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerID="e5be6121d8f4c94f8d903919ec701b151040829d75fa0df410754894f515f972" exitCode=0
Sep 11 04:44:41.389550 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:41.389238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerDied","Data":"e5be6121d8f4c94f8d903919ec701b151040829d75fa0df410754894f515f972"}
Sep 11 04:44:42.532559 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.532523    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:42.623957 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.623921    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjfqg\" (UniqueName: \"kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg\") pod \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") "
Sep 11 04:44:42.624121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.623989    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle\") pod \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") "
Sep 11 04:44:42.624121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.624044    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util\") pod \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\" (UID: \"67aedb4f-9b56-4882-bc5f-4908c1a98c50\") "
Sep 11 04:44:42.624728 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.624662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle" (OuterVolumeSpecName: "bundle") pod "67aedb4f-9b56-4882-bc5f-4908c1a98c50" (UID: "67aedb4f-9b56-4882-bc5f-4908c1a98c50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:44:42.626275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.626253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg" (OuterVolumeSpecName: "kube-api-access-fjfqg") pod "67aedb4f-9b56-4882-bc5f-4908c1a98c50" (UID: "67aedb4f-9b56-4882-bc5f-4908c1a98c50"). InnerVolumeSpecName "kube-api-access-fjfqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:44:42.630098 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.630071    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util" (OuterVolumeSpecName: "util") pod "67aedb4f-9b56-4882-bc5f-4908c1a98c50" (UID: "67aedb4f-9b56-4882-bc5f-4908c1a98c50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:44:42.725285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.725254    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:42.725285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.725280    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67aedb4f-9b56-4882-bc5f-4908c1a98c50-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:42.725285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:42.725289    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjfqg\" (UniqueName: \"kubernetes.io/projected/67aedb4f-9b56-4882-bc5f-4908c1a98c50-kube-api-access-fjfqg\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:44:43.397614 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:43.397568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262" event={"ID":"67aedb4f-9b56-4882-bc5f-4908c1a98c50","Type":"ContainerDied","Data":"bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159"}
Sep 11 04:44:43.397781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:43.397625    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408f262"
Sep 11 04:44:43.397781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:43.397629    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bea10dab37195af64c4eeecd424ba500ca77f38179412943e39f4b52d17b3159"
Sep 11 04:44:54.531472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531445    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"]
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531702    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="pull"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531715    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="pull"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531722    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="extract"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531729    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="extract"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531744    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="util"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531748    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="util"
Sep 11 04:44:54.531954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.531793    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67aedb4f-9b56-4882-bc5f-4908c1a98c50" containerName="extract"
Sep 11 04:44:54.535720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.535704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.538107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.538083    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 04:44:54.538107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.538099    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 04:44:54.538795 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.538782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzm5d\""
Sep 11 04:44:54.547061 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.547040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"]
Sep 11 04:44:54.620775 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.620733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.620775 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.620773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.620998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.620790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wprjx\" (UniqueName: \"kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.722013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.721971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.722013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.722014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.722301 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.722039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wprjx\" (UniqueName: \"kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.722361 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.722348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.722412 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.722373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.731413 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.731387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wprjx\" (UniqueName: \"kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:54.844286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:54.844183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:44:55.115950 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:55.115925    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"]
Sep 11 04:44:55.118377 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:44:55.118346    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b638a5_6604_465f_aa1c_6f42f0769d26.slice/crio-1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c WatchSource:0}: Error finding container 1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c: Status 404 returned error can't find the container with id 1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c
Sep 11 04:44:55.442306 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:55.442205    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerID="d527e9a71b6061b26c11a311e1b2c6086467ac4d0fa3fe0d0c751059f4ec57b7" exitCode=0
Sep 11 04:44:55.442306 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:55.442285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9" event={"ID":"d1b638a5-6604-465f-aa1c-6f42f0769d26","Type":"ContainerDied","Data":"d527e9a71b6061b26c11a311e1b2c6086467ac4d0fa3fe0d0c751059f4ec57b7"}
Sep 11 04:44:55.442496 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:55.442357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9" event={"ID":"d1b638a5-6604-465f-aa1c-6f42f0769d26","Type":"ContainerStarted","Data":"1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c"}
Sep 11 04:44:58.457170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:58.457136    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerID="93e2939d4b96647efa8004ef738c7b19100813ff8146eca2747bed0a4fe90341" exitCode=0
Sep 11 04:44:58.457527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:58.457203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9" event={"ID":"d1b638a5-6604-465f-aa1c-6f42f0769d26","Type":"ContainerDied","Data":"93e2939d4b96647efa8004ef738c7b19100813ff8146eca2747bed0a4fe90341"}
Sep 11 04:44:59.463728 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:59.463697    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerID="84af0eff0921d8f14d7419e539bc79e56ae28c61738266a3c5d030ef71397890" exitCode=0
Sep 11 04:44:59.464083 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:44:59.463746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9" event={"ID":"d1b638a5-6604-465f-aa1c-6f42f0769d26","Type":"ContainerDied","Data":"84af0eff0921d8f14d7419e539bc79e56ae28c61738266a3c5d030ef71397890"}
Sep 11 04:45:00.587182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.587155    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:45:00.671096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.671069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util\") pod \"d1b638a5-6604-465f-aa1c-6f42f0769d26\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") "
Sep 11 04:45:00.671243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.671116    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle\") pod \"d1b638a5-6604-465f-aa1c-6f42f0769d26\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") "
Sep 11 04:45:00.671243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.671163    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wprjx\" (UniqueName: \"kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx\") pod \"d1b638a5-6604-465f-aa1c-6f42f0769d26\" (UID: \"d1b638a5-6604-465f-aa1c-6f42f0769d26\") "
Sep 11 04:45:00.671511 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.671493    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle" (OuterVolumeSpecName: "bundle") pod "d1b638a5-6604-465f-aa1c-6f42f0769d26" (UID: "d1b638a5-6604-465f-aa1c-6f42f0769d26"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:00.673386 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.673362    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx" (OuterVolumeSpecName: "kube-api-access-wprjx") pod "d1b638a5-6604-465f-aa1c-6f42f0769d26" (UID: "d1b638a5-6604-465f-aa1c-6f42f0769d26"). InnerVolumeSpecName "kube-api-access-wprjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:45:00.678022 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.677983    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util" (OuterVolumeSpecName: "util") pod "d1b638a5-6604-465f-aa1c-6f42f0769d26" (UID: "d1b638a5-6604-465f-aa1c-6f42f0769d26"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:00.771955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.771862    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:00.771955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.771904    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1b638a5-6604-465f-aa1c-6f42f0769d26-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:00.771955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:00.771914    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wprjx\" (UniqueName: \"kubernetes.io/projected/d1b638a5-6604-465f-aa1c-6f42f0769d26-kube-api-access-wprjx\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:01.472291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.472245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9" event={"ID":"d1b638a5-6604-465f-aa1c-6f42f0769d26","Type":"ContainerDied","Data":"1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c"}
Sep 11 04:45:01.472291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.472287    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5lcj9"
Sep 11 04:45:01.472486 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.472302    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c2415f969112e6c6a91dbb79ade60065c3c1a0b67abeb18783f8b05384c016c"
Sep 11 04:45:01.827815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.827735    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"]
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.827992    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="pull"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828007    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="pull"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828026    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="extract"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828031    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="extract"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828041    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="util"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828046    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="util"
Sep 11 04:45:01.828146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.828106    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b638a5-6604-465f-aa1c-6f42f0769d26" containerName="extract"
Sep 11 04:45:01.831552 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.831537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:01.833662 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.833634    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 04:45:01.833799 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.833672    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-h5xdt\""
Sep 11 04:45:01.833799 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.833713    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 04:45:01.839365 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.839339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"]
Sep 11 04:45:01.980547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.980505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:01.980730 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:01.980563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjlw8\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-kube-api-access-rjlw8\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.081613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.081504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjlw8\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-kube-api-access-rjlw8\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.081613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.081599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.089444 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.089415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.089566 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.089479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjlw8\" (UniqueName: \"kubernetes.io/projected/08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b-kube-api-access-rjlw8\") pod \"cert-manager-cainjector-559cdd8c87-sq6jg\" (UID: \"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.140498 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.140462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"
Sep 11 04:45:02.412888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.412865    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg"]
Sep 11 04:45:02.414960 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:02.414936    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08b5abf6_b3aa_46cb_9edd_8dfcb6f7215b.slice/crio-148aeafbf6731b8c264df1b7fc7a1a1cf563fcc6a72a0e346529a930ab06b787 WatchSource:0}: Error finding container 148aeafbf6731b8c264df1b7fc7a1a1cf563fcc6a72a0e346529a930ab06b787: Status 404 returned error can't find the container with id 148aeafbf6731b8c264df1b7fc7a1a1cf563fcc6a72a0e346529a930ab06b787
Sep 11 04:45:02.477138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:02.477099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg" event={"ID":"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b","Type":"ContainerStarted","Data":"148aeafbf6731b8c264df1b7fc7a1a1cf563fcc6a72a0e346529a930ab06b787"}
Sep 11 04:45:05.497142 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:05.497087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg" event={"ID":"08b5abf6-b3aa-46cb-9edd-8dfcb6f7215b","Type":"ContainerStarted","Data":"879d3cba70044e4f156687cc5d585e0d3e396b5caed988e51c75931fb07b1d00"}
Sep 11 04:45:05.511759 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:05.511680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sq6jg" podStartSLOduration=1.5584938419999999 podStartE2EDuration="4.511662908s" totalImagesPullingTime="2.953169066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:05.511243137 +0000 UTC m=+299.802717577" watchObservedRunningTime="2026-09-11 04:45:05.511662908 +0000 UTC m=+299.803137346"
Sep 11 04:45:06.088897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:06.088873    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:45:06.091093 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:06.091072    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:45:06.100126 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:06.100107    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 04:45:07.109451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.109418    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"]
Sep 11 04:45:07.168981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.168961    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"]
Sep 11 04:45:07.169090 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.169057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.171285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.171269    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cqkvs\""
Sep 11 04:45:07.171387 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.171308    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 04:45:07.171822 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.171805    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:45:07.324452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.324422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c507738-ded7-4541-9fe0-1cfc623064af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.324663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.324469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5xrh\" (UniqueName: \"kubernetes.io/projected/4c507738-ded7-4541-9fe0-1cfc623064af-kube-api-access-m5xrh\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.425882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.425805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c507738-ded7-4541-9fe0-1cfc623064af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.425882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.425843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5xrh\" (UniqueName: \"kubernetes.io/projected/4c507738-ded7-4541-9fe0-1cfc623064af-kube-api-access-m5xrh\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.426180 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.426161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c507738-ded7-4541-9fe0-1cfc623064af-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.433423 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.433402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5xrh\" (UniqueName: \"kubernetes.io/projected/4c507738-ded7-4541-9fe0-1cfc623064af-kube-api-access-m5xrh\") pod \"openshift-lws-operator-fd8ccff4c-4hnrk\" (UID: \"4c507738-ded7-4541-9fe0-1cfc623064af\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.478430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.478407    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"
Sep 11 04:45:07.767899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.767872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk"]
Sep 11 04:45:07.770266 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:07.770236    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c507738_ded7_4541_9fe0_1cfc623064af.slice/crio-b92e01e40f2f434658a565316ee4cd76be2c2de4ee39a10a62da7281f5866f36 WatchSource:0}: Error finding container b92e01e40f2f434658a565316ee4cd76be2c2de4ee39a10a62da7281f5866f36: Status 404 returned error can't find the container with id b92e01e40f2f434658a565316ee4cd76be2c2de4ee39a10a62da7281f5866f36
Sep 11 04:45:07.772239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:07.772219    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:45:08.510114 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:08.510077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk" event={"ID":"4c507738-ded7-4541-9fe0-1cfc623064af","Type":"ContainerStarted","Data":"b92e01e40f2f434658a565316ee4cd76be2c2de4ee39a10a62da7281f5866f36"}
Sep 11 04:45:11.524051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:11.524015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk" event={"ID":"4c507738-ded7-4541-9fe0-1cfc623064af","Type":"ContainerStarted","Data":"5084220025e72b4e68b51b4445c1441657bd42a8317a84327e6dac2a4086882d"}
Sep 11 04:45:11.540126 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:11.540074    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hnrk" podStartSLOduration=1.733297624 podStartE2EDuration="4.540059593s" totalImagesPullingTime="2.806761969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:11.538609539 +0000 UTC m=+305.830083977" watchObservedRunningTime="2026-09-11 04:45:11.540059593 +0000 UTC m=+305.831534029"
Sep 11 04:45:14.297864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.297828    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"]
Sep 11 04:45:14.300079 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.300046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.302236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.302215    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 04:45:14.302348 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.302215    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 04:45:14.302870 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.302853    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzm5d\""
Sep 11 04:45:14.308355 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.308334    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"]
Sep 11 04:45:14.379284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.379253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvxr4\" (UniqueName: \"kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.379434 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.379294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.379434 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.379325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.479860 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.479832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvxr4\" (UniqueName: \"kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.479977 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.479875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.479977 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.479902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.480265 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.480247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.480265 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.480257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.487848 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.487827    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvxr4\" (UniqueName: \"kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.609784 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.609709    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:14.878874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:14.878849    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"]
Sep 11 04:45:14.881196 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:14.881171    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode778b565_98f0_4d76_8b94_2da86c62d258.slice/crio-4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c WatchSource:0}: Error finding container 4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c: Status 404 returned error can't find the container with id 4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c
Sep 11 04:45:15.543071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:15.543040    2581 generic.go:356] "Generic (PLEG): container finished" podID="e778b565-98f0-4d76-8b94-2da86c62d258" containerID="fa4395fa50e22ea58f63a0f84d2fb105de2525c0364b0fca1a9995174819010d" exitCode=0
Sep 11 04:45:15.543453 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:15.543100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg" event={"ID":"e778b565-98f0-4d76-8b94-2da86c62d258","Type":"ContainerDied","Data":"fa4395fa50e22ea58f63a0f84d2fb105de2525c0364b0fca1a9995174819010d"}
Sep 11 04:45:15.543453 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:15.543136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg" event={"ID":"e778b565-98f0-4d76-8b94-2da86c62d258","Type":"ContainerStarted","Data":"4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c"}
Sep 11 04:45:16.549104 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:16.549071    2581 generic.go:356] "Generic (PLEG): container finished" podID="e778b565-98f0-4d76-8b94-2da86c62d258" containerID="648b9bdcbc2d4a9f35b33c40a74868f3e76b7982b1ff1c5d07630c92582768ee" exitCode=0
Sep 11 04:45:16.549493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:16.549159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg" event={"ID":"e778b565-98f0-4d76-8b94-2da86c62d258","Type":"ContainerDied","Data":"648b9bdcbc2d4a9f35b33c40a74868f3e76b7982b1ff1c5d07630c92582768ee"}
Sep 11 04:45:17.555886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:17.555855    2581 generic.go:356] "Generic (PLEG): container finished" podID="e778b565-98f0-4d76-8b94-2da86c62d258" containerID="29678ff152fe6b4f5a9641de08f7208b4fd9389d1fdfc2215e9e6955e27bfec4" exitCode=0
Sep 11 04:45:17.556245 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:17.555935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg" event={"ID":"e778b565-98f0-4d76-8b94-2da86c62d258","Type":"ContainerDied","Data":"29678ff152fe6b4f5a9641de08f7208b4fd9389d1fdfc2215e9e6955e27bfec4"}
Sep 11 04:45:18.676762 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.676736    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:18.705502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.705468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvxr4\" (UniqueName: \"kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4\") pod \"e778b565-98f0-4d76-8b94-2da86c62d258\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") "
Sep 11 04:45:18.705680 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.705509    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util\") pod \"e778b565-98f0-4d76-8b94-2da86c62d258\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") "
Sep 11 04:45:18.705680 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.705541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle\") pod \"e778b565-98f0-4d76-8b94-2da86c62d258\" (UID: \"e778b565-98f0-4d76-8b94-2da86c62d258\") "
Sep 11 04:45:18.706378 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.706345    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle" (OuterVolumeSpecName: "bundle") pod "e778b565-98f0-4d76-8b94-2da86c62d258" (UID: "e778b565-98f0-4d76-8b94-2da86c62d258"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:18.708033 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.708005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4" (OuterVolumeSpecName: "kube-api-access-hvxr4") pod "e778b565-98f0-4d76-8b94-2da86c62d258" (UID: "e778b565-98f0-4d76-8b94-2da86c62d258"). InnerVolumeSpecName "kube-api-access-hvxr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:45:18.711267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.711248    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util" (OuterVolumeSpecName: "util") pod "e778b565-98f0-4d76-8b94-2da86c62d258" (UID: "e778b565-98f0-4d76-8b94-2da86c62d258"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:18.806536 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.806502    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvxr4\" (UniqueName: \"kubernetes.io/projected/e778b565-98f0-4d76-8b94-2da86c62d258-kube-api-access-hvxr4\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:18.806536 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.806533    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:18.806744 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:18.806545    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e778b565-98f0-4d76-8b94-2da86c62d258-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:19.565996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:19.565959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg" event={"ID":"e778b565-98f0-4d76-8b94-2da86c62d258","Type":"ContainerDied","Data":"4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c"}
Sep 11 04:45:19.566154 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:19.566011    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4324c3b3a5b27908ac3ba5c61b7a558a82874967678ff3e6051500f9f21adc1c"
Sep 11 04:45:19.566154 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:19.565970    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxh7zg"
Sep 11 04:45:24.714187 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714147    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"]
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714522    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="util"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714539    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="util"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714561    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="extract"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714570    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="extract"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714611    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="pull"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714619    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="pull"
Sep 11 04:45:24.714706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.714685    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e778b565-98f0-4d76-8b94-2da86c62d258" containerName="extract"
Sep 11 04:45:24.717451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.717431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.719517 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.719498    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzm5d\""
Sep 11 04:45:24.719636 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.719525    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 04:45:24.720222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.720207    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 04:45:24.725773 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.725744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"]
Sep 11 04:45:24.744484 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.744453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.744610 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.744497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6lm\" (UniqueName: \"kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.744610 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.744563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.844935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.844906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6lm\" (UniqueName: \"kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.845080 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.844954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.845080 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.844992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.845396 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.845373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.845460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.845382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:24.857435 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:24.857414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6lm\" (UniqueName: \"kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:25.027499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:25.027411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:25.299948 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:25.299924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"]
Sep 11 04:45:25.301878 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:25.301855    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126815fb_1fe7_48a8_9d4f_ba446aae84fc.slice/crio-52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467 WatchSource:0}: Error finding container 52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467: Status 404 returned error can't find the container with id 52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467
Sep 11 04:45:25.590542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:25.590458    2581 generic.go:356] "Generic (PLEG): container finished" podID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerID="b848d163f513624d7db539157262a459c00d35f6aa5d00bb7027f196216c0c43" exitCode=0
Sep 11 04:45:25.590542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:25.590506    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4" event={"ID":"126815fb-1fe7-48a8-9d4f-ba446aae84fc","Type":"ContainerDied","Data":"b848d163f513624d7db539157262a459c00d35f6aa5d00bb7027f196216c0c43"}
Sep 11 04:45:25.590780 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:25.590556    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4" event={"ID":"126815fb-1fe7-48a8-9d4f-ba446aae84fc","Type":"ContainerStarted","Data":"52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467"}
Sep 11 04:45:26.596313 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.596278    2581 generic.go:356] "Generic (PLEG): container finished" podID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerID="9c5d5077e748fd02d09942cee041d79d80a571674c4d411ab50290720657b030" exitCode=0
Sep 11 04:45:26.596709 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.596327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4" event={"ID":"126815fb-1fe7-48a8-9d4f-ba446aae84fc","Type":"ContainerDied","Data":"9c5d5077e748fd02d09942cee041d79d80a571674c4d411ab50290720657b030"}
Sep 11 04:45:26.669434 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.669406    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"]
Sep 11 04:45:26.671531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.671516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.673742 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.673718    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 04:45:26.673861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.673762    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 04:45:26.674085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.674061    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 04:45:26.674183 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.674164    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 04:45:26.674234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.674204    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4g22g\""
Sep 11 04:45:26.692026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.691999    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"]
Sep 11 04:45:26.758083 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.758059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.758191 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.758107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.758235 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.758202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.758269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.758231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.758269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.758250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgmj\" (UniqueName: \"kubernetes.io/projected/e055a6bb-df5d-425c-96c8-fa65d4618e38-kube-api-access-nhgmj\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.858908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.858819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.858908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.858858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgmj\" (UniqueName: \"kubernetes.io/projected/e055a6bb-df5d-425c-96c8-fa65d4618e38-kube-api-access-nhgmj\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.858908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.858901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.859182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.858952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.859182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.859006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.859312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.859292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.861131 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.861106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e055a6bb-df5d-425c-96c8-fa65d4618e38-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.861358 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.861343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.861512 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.861493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e055a6bb-df5d-425c-96c8-fa65d4618e38-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.866596 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.866550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgmj\" (UniqueName: \"kubernetes.io/projected/e055a6bb-df5d-425c-96c8-fa65d4618e38-kube-api-access-nhgmj\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm\" (UID: \"e055a6bb-df5d-425c-96c8-fa65d4618e38\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:26.982737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:26.982703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:27.256095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:27.255800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"]
Sep 11 04:45:27.258519 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:27.258483    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode055a6bb_df5d_425c_96c8_fa65d4618e38.slice/crio-0f34c46c8b64552cce0456b282316d79c40acf3e429cccbaa3a5b97659b42b66 WatchSource:0}: Error finding container 0f34c46c8b64552cce0456b282316d79c40acf3e429cccbaa3a5b97659b42b66: Status 404 returned error can't find the container with id 0f34c46c8b64552cce0456b282316d79c40acf3e429cccbaa3a5b97659b42b66
Sep 11 04:45:27.602103 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:27.602015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm" event={"ID":"e055a6bb-df5d-425c-96c8-fa65d4618e38","Type":"ContainerStarted","Data":"0f34c46c8b64552cce0456b282316d79c40acf3e429cccbaa3a5b97659b42b66"}
Sep 11 04:45:27.604252 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:27.604228    2581 generic.go:356] "Generic (PLEG): container finished" podID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerID="85608b41e23457ec15d64a3d1d51fcdb11c8fa9b82a58b8e5fb9aa11aa480123" exitCode=0
Sep 11 04:45:27.604363 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:27.604279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4" event={"ID":"126815fb-1fe7-48a8-9d4f-ba446aae84fc","Type":"ContainerDied","Data":"85608b41e23457ec15d64a3d1d51fcdb11c8fa9b82a58b8e5fb9aa11aa480123"}
Sep 11 04:45:30.232366 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.232333    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:30.286010 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.285970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle\") pod \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") "
Sep 11 04:45:30.286128 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.286038    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util\") pod \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") "
Sep 11 04:45:30.286128 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.286065    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn6lm\" (UniqueName: \"kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm\") pod \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\" (UID: \"126815fb-1fe7-48a8-9d4f-ba446aae84fc\") "
Sep 11 04:45:30.286896 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.286865    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle" (OuterVolumeSpecName: "bundle") pod "126815fb-1fe7-48a8-9d4f-ba446aae84fc" (UID: "126815fb-1fe7-48a8-9d4f-ba446aae84fc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:30.288120 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.288101    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm" (OuterVolumeSpecName: "kube-api-access-cn6lm") pod "126815fb-1fe7-48a8-9d4f-ba446aae84fc" (UID: "126815fb-1fe7-48a8-9d4f-ba446aae84fc"). InnerVolumeSpecName "kube-api-access-cn6lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:45:30.291352 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.291332    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util" (OuterVolumeSpecName: "util") pod "126815fb-1fe7-48a8-9d4f-ba446aae84fc" (UID: "126815fb-1fe7-48a8-9d4f-ba446aae84fc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:45:30.386858 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.386812    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:30.386858 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.386854    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn6lm\" (UniqueName: \"kubernetes.io/projected/126815fb-1fe7-48a8-9d4f-ba446aae84fc-kube-api-access-cn6lm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:30.386858 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.386865    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/126815fb-1fe7-48a8-9d4f-ba446aae84fc-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:45:30.619949 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.619860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm" event={"ID":"e055a6bb-df5d-425c-96c8-fa65d4618e38","Type":"ContainerStarted","Data":"229a33596f9a883b5f4a20eebbd3e393d19675986a214e9597d84e624a928124"}
Sep 11 04:45:30.621839 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.621819    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4"
Sep 11 04:45:30.621978 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.621842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mwnd4" event={"ID":"126815fb-1fe7-48a8-9d4f-ba446aae84fc","Type":"ContainerDied","Data":"52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467"}
Sep 11 04:45:30.621978 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:30.621875    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52d84d2c5899b281a90cde32e170c3e49caa62f8f49f728894700f827b05e467"
Sep 11 04:45:31.627333 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:31.627246    2581 generic.go:356] "Generic (PLEG): container finished" podID="e055a6bb-df5d-425c-96c8-fa65d4618e38" containerID="229a33596f9a883b5f4a20eebbd3e393d19675986a214e9597d84e624a928124" exitCode=0
Sep 11 04:45:31.627333 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:31.627293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm" event={"ID":"e055a6bb-df5d-425c-96c8-fa65d4618e38","Type":"ContainerDied","Data":"229a33596f9a883b5f4a20eebbd3e393d19675986a214e9597d84e624a928124"}
Sep 11 04:45:32.633988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:32.633954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm" event={"ID":"e055a6bb-df5d-425c-96c8-fa65d4618e38","Type":"ContainerStarted","Data":"67c97dfe1325d3e0b71bf6dd9946f9f09fb04b1244d375bf7154c3805848cfc6"}
Sep 11 04:45:32.634371 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:32.634056    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:32.656646 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:32.656596    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm" podStartSLOduration=2.3307418 podStartE2EDuration="6.6565603s" totalImagesPullingTime="3.318882108s" totalInitContainerRuntime="1.006936392s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:32.654886291 +0000 UTC m=+326.946360728" watchObservedRunningTime="2026-09-11 04:45:32.6565603 +0000 UTC m=+326.948034737"
Sep 11 04:45:43.640241 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:43.640213    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm"
Sep 11 04:45:46.132971 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.132939    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"]
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133184    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="extract"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133194    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="extract"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133204    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="util"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133209    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="util"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133218    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="pull"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133223    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="pull"
Sep 11 04:45:46.133369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.133285    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="126815fb-1fe7-48a8-9d4f-ba446aae84fc" containerName="extract"
Sep 11 04:45:46.142569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.142549    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.160722 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.160700    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 04:45:46.160831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.160700    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 04:45:46.161000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.160982    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 04:45:46.161346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.161328    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 04:45:46.161452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.161363    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fsr94\""
Sep 11 04:45:46.181237 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.181203    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"]
Sep 11 04:45:46.210123 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.210089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942af10e-fe96-4859-be53-7f7b3e9f5407-tls-certs\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.210279 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.210148    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942af10e-fe96-4859-be53-7f7b3e9f5407-tmp\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.210279 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.210198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfzh6\" (UniqueName: \"kubernetes.io/projected/942af10e-fe96-4859-be53-7f7b3e9f5407-kube-api-access-rfzh6\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.311564 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.311531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfzh6\" (UniqueName: \"kubernetes.io/projected/942af10e-fe96-4859-be53-7f7b3e9f5407-kube-api-access-rfzh6\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.311771 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.311607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942af10e-fe96-4859-be53-7f7b3e9f5407-tls-certs\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.311771 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.311666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942af10e-fe96-4859-be53-7f7b3e9f5407-tmp\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.314139 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.314108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/942af10e-fe96-4859-be53-7f7b3e9f5407-tmp\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.314290 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.314274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942af10e-fe96-4859-be53-7f7b3e9f5407-tls-certs\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.319168 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.319140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfzh6\" (UniqueName: \"kubernetes.io/projected/942af10e-fe96-4859-be53-7f7b3e9f5407-kube-api-access-rfzh6\") pod \"kube-auth-proxy-fcb5f7d79-d492c\" (UID: \"942af10e-fe96-4859-be53-7f7b3e9f5407\") " pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.451422 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.451392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"
Sep 11 04:45:46.724367 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:46.724343    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c"]
Sep 11 04:45:46.726287 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:46.726260    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942af10e_fe96_4859_be53_7f7b3e9f5407.slice/crio-3a2547dfba0e0176495caedf582400051b7656a69dd91e6e7d4afb73ef397116 WatchSource:0}: Error finding container 3a2547dfba0e0176495caedf582400051b7656a69dd91e6e7d4afb73ef397116: Status 404 returned error can't find the container with id 3a2547dfba0e0176495caedf582400051b7656a69dd91e6e7d4afb73ef397116
Sep 11 04:45:47.698988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:47.698951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c" event={"ID":"942af10e-fe96-4859-be53-7f7b3e9f5407","Type":"ContainerStarted","Data":"3a2547dfba0e0176495caedf582400051b7656a69dd91e6e7d4afb73ef397116"}
Sep 11 04:45:50.713910 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:50.713874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c" event={"ID":"942af10e-fe96-4859-be53-7f7b3e9f5407","Type":"ContainerStarted","Data":"a6f7c415e1c8d40ba59b7fbd911b3e16b8be4773c314c59f4b0e5048b2bd0bcd"}
Sep 11 04:45:50.731219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:50.731174    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-fcb5f7d79-d492c" podStartSLOduration=1.6094326209999998 podStartE2EDuration="4.731158914s" totalImagesPullingTime="3.121726293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:50.730221082 +0000 UTC m=+345.021695520" watchObservedRunningTime="2026-09-11 04:45:50.731158914 +0000 UTC m=+345.022633351"
Sep 11 04:45:52.812434 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.812403    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"]
Sep 11 04:45:52.815627 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.815607    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.818641 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.818616    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 04:45:52.818822 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.818803    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-m4vqw\""
Sep 11 04:45:52.829309 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.829288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"]
Sep 11 04:45:52.860843 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.860817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hb52\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-kube-api-access-6hb52\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.860954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.860847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.860954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.860869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.860954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.860885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.861057 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.860980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.861057 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.861012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.861057 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.861037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.861147 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.861090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.861147 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.861122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.961835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.961804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.961835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.961841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hb52\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-kube-api-access-6hb52\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.961859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.961976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962594 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962561    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962705 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.962705 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.962617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.964345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.964327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.964678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.964660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.972531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.972504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:52.972719 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:52.972701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hb52\" (UniqueName: \"kubernetes.io/projected/2769ea72-85d6-44d3-ad7d-dae206e9cf47-kube-api-access-6hb52\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg\" (UID: \"2769ea72-85d6-44d3-ad7d-dae206e9cf47\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:53.126196 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:53.126123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:53.401608 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:53.401555    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"]
Sep 11 04:45:53.404321 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:53.404293    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2769ea72_85d6_44d3_ad7d_dae206e9cf47.slice/crio-f02d12d4dd7052750b6c7256137d1ba9c893837872baa97f7135cc44baeb4ec5 WatchSource:0}: Error finding container f02d12d4dd7052750b6c7256137d1ba9c893837872baa97f7135cc44baeb4ec5: Status 404 returned error can't find the container with id f02d12d4dd7052750b6c7256137d1ba9c893837872baa97f7135cc44baeb4ec5
Sep 11 04:45:53.727729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:53.727699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg" event={"ID":"2769ea72-85d6-44d3-ad7d-dae206e9cf47","Type":"ContainerStarted","Data":"f02d12d4dd7052750b6c7256137d1ba9c893837872baa97f7135cc44baeb4ec5"}
Sep 11 04:45:56.313120 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.313072    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:45:56.313419 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.313149    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:45:56.313419 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.313186    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:45:56.744634 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.744597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg" event={"ID":"2769ea72-85d6-44d3-ad7d-dae206e9cf47","Type":"ContainerStarted","Data":"79977b416a25789322d3b4c28ba3fe856a88d18f36c0493bf246725fb3a63a83"}
Sep 11 04:45:56.766894 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.766839    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg" podStartSLOduration=1.860361665 podStartE2EDuration="4.766824289s" totalImagesPullingTime="2.906462624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:56.76540173 +0000 UTC m=+351.056876170" watchObservedRunningTime="2026-09-11 04:45:56.766824289 +0000 UTC m=+351.058298726"
Sep 11 04:45:56.774737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.774712    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76qwm"]
Sep 11 04:45:56.778317 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.778298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:56.780308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.780284    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 04:45:56.780308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.780303    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kp7cn\""
Sep 11 04:45:56.786798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.786750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76qwm"]
Sep 11 04:45:56.895352 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.895322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf62e86-0a85-409c-86ea-badf16584a4d-cert\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:56.895512 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.895371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcj2h\" (UniqueName: \"kubernetes.io/projected/3bf62e86-0a85-409c-86ea-badf16584a4d-kube-api-access-tcj2h\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:56.996068 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.995991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcj2h\" (UniqueName: \"kubernetes.io/projected/3bf62e86-0a85-409c-86ea-badf16584a4d-kube-api-access-tcj2h\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:56.996068 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.996050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf62e86-0a85-409c-86ea-badf16584a4d-cert\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:56.998504 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:56.998474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf62e86-0a85-409c-86ea-badf16584a4d-cert\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:57.003022 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.003002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcj2h\" (UniqueName: \"kubernetes.io/projected/3bf62e86-0a85-409c-86ea-badf16584a4d-kube-api-access-tcj2h\") pod \"odh-model-controller-7998c67dc4-76qwm\" (UID: \"3bf62e86-0a85-409c-86ea-badf16584a4d\") " pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:57.090335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.090298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:45:57.126546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.126513    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:57.131862 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.131831    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:57.303872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.303796    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjwcz"]
Sep 11 04:45:57.308253 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.308238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.310345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.310327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 04:45:57.310501 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.310487    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9c67f\""
Sep 11 04:45:57.324403 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.324376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjwcz"]
Sep 11 04:45:57.367399 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.367368    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76qwm"]
Sep 11 04:45:57.371041 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:57.371013    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf62e86_0a85_409c_86ea_badf16584a4d.slice/crio-69a9f27f52e47731b736f30ca53d0d9680852130ebfdd016e7c4b0f4952a9512 WatchSource:0}: Error finding container 69a9f27f52e47731b736f30ca53d0d9680852130ebfdd016e7c4b0f4952a9512: Status 404 returned error can't find the container with id 69a9f27f52e47731b736f30ca53d0d9680852130ebfdd016e7c4b0f4952a9512
Sep 11 04:45:57.399078 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.399053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj5xr\" (UniqueName: \"kubernetes.io/projected/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-kube-api-access-zj5xr\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.399199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.399119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.500383 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.500347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj5xr\" (UniqueName: \"kubernetes.io/projected/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-kube-api-access-zj5xr\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.500558 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.500408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.502902 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.502885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.507867 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.507841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj5xr\" (UniqueName: \"kubernetes.io/projected/6434134a-0b1f-4c7f-814c-0ecdd3aeec5d-kube-api-access-zj5xr\") pod \"model-serving-api-54d89bd5fc-qjwcz\" (UID: \"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.625805 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.625708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:45:57.750021 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.749944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm" event={"ID":"3bf62e86-0a85-409c-86ea-badf16584a4d","Type":"ContainerStarted","Data":"69a9f27f52e47731b736f30ca53d0d9680852130ebfdd016e7c4b0f4952a9512"}
Sep 11 04:45:57.750281 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.750260    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:57.751323 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.751305    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg"
Sep 11 04:45:57.900541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:57.900521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjwcz"]
Sep 11 04:45:57.902745 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:45:57.902715    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6434134a_0b1f_4c7f_814c_0ecdd3aeec5d.slice/crio-ee1a15a97447dc956c39c1b31fe932b463ca08ee14927aa30a69a6ae79242247 WatchSource:0}: Error finding container ee1a15a97447dc956c39c1b31fe932b463ca08ee14927aa30a69a6ae79242247: Status 404 returned error can't find the container with id ee1a15a97447dc956c39c1b31fe932b463ca08ee14927aa30a69a6ae79242247
Sep 11 04:45:58.759182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:45:58.759095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz" event={"ID":"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d","Type":"ContainerStarted","Data":"ee1a15a97447dc956c39c1b31fe932b463ca08ee14927aa30a69a6ae79242247"}
Sep 11 04:46:01.778042 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.778006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm" event={"ID":"3bf62e86-0a85-409c-86ea-badf16584a4d","Type":"ContainerStarted","Data":"2bcae71e78fb3fc919175addaca8164defd0247f66d78e4fc71cc9bd048a9dc5"}
Sep 11 04:46:01.778497 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.778096    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:46:01.779680 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.779650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz" event={"ID":"6434134a-0b1f-4c7f-814c-0ecdd3aeec5d","Type":"ContainerStarted","Data":"15e69bffde0164ad8a695f14e64d664d97d868cf5e5eca411120c6cf42ab807d"}
Sep 11 04:46:01.779798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.779769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:46:01.793970 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.793921    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm" podStartSLOduration=2.4118554850000002 podStartE2EDuration="5.793907714s" totalImagesPullingTime="3.382052229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:46:01.791733688 +0000 UTC m=+356.083208125" watchObservedRunningTime="2026-09-11 04:46:01.793907714 +0000 UTC m=+356.085382150"
Sep 11 04:46:01.806538 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:01.806492    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz" podStartSLOduration=1.96637873 podStartE2EDuration="4.806476481s" totalImagesPullingTime="2.840097751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:46:01.806281623 +0000 UTC m=+356.097756061" watchObservedRunningTime="2026-09-11 04:46:01.806476481 +0000 UTC m=+356.097950920"
Sep 11 04:46:05.120677 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.120635    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"]
Sep 11 04:46:05.131206 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.131180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.132913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.132889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"]
Sep 11 04:46:05.133676 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.133658    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6dbkv\""
Sep 11 04:46:05.134552 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.134531    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 04:46:05.134717 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.134613    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 04:46:05.134797 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.134653    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 04:46:05.266856 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.266830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.266954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.266859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.266995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.266951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0344db76-659e-4118-9702-564aa962fd50-manager-config\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.267029 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.267002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcrgj\" (UniqueName: \"kubernetes.io/projected/0344db76-659e-4118-9702-564aa962fd50-kube-api-access-tcrgj\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.367784 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.367758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcrgj\" (UniqueName: \"kubernetes.io/projected/0344db76-659e-4118-9702-564aa962fd50-kube-api-access-tcrgj\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.367890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.367792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.367890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.367816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.367890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.367875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0344db76-659e-4118-9702-564aa962fd50-manager-config\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.368495 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.368474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0344db76-659e-4118-9702-564aa962fd50-manager-config\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.370562 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.370533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.370728 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.370542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0344db76-659e-4118-9702-564aa962fd50-cert\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.378369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.378344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcrgj\" (UniqueName: \"kubernetes.io/projected/0344db76-659e-4118-9702-564aa962fd50-kube-api-access-tcrgj\") pod \"lws-controller-manager-6f84bb6f84-bwdnj\" (UID: \"0344db76-659e-4118-9702-564aa962fd50\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.441343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.441306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:05.717654 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.717630    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"]
Sep 11 04:46:05.719682 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:46:05.719655    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0344db76_659e_4118_9702_564aa962fd50.slice/crio-e5483c73e8f406335f92cc0fd4d4696bf1c7fba4a62f3f12ad9a1621d8e072cf WatchSource:0}: Error finding container e5483c73e8f406335f92cc0fd4d4696bf1c7fba4a62f3f12ad9a1621d8e072cf: Status 404 returned error can't find the container with id e5483c73e8f406335f92cc0fd4d4696bf1c7fba4a62f3f12ad9a1621d8e072cf
Sep 11 04:46:05.799830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:05.799798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj" event={"ID":"0344db76-659e-4118-9702-564aa962fd50","Type":"ContainerStarted","Data":"e5483c73e8f406335f92cc0fd4d4696bf1c7fba4a62f3f12ad9a1621d8e072cf"}
Sep 11 04:46:08.816085 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:08.816049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj" event={"ID":"0344db76-659e-4118-9702-564aa962fd50","Type":"ContainerStarted","Data":"339a7df526939d23ff4def26319573e658203976466c70884ad13701ea4b2be7"}
Sep 11 04:46:08.816468 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:08.816192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:46:08.850282 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:08.850231    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj" podStartSLOduration=1.411006934 podStartE2EDuration="3.850214932s" totalImagesPullingTime="2.439207998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:46:08.848211549 +0000 UTC m=+363.139685987" watchObservedRunningTime="2026-09-11 04:46:08.850214932 +0000 UTC m=+363.141689368"
Sep 11 04:46:12.787363 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:12.787335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-76qwm"
Sep 11 04:46:12.789322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:12.789300    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qjwcz"
Sep 11 04:46:19.823257 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:46:19.823228    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-bwdnj"
Sep 11 04:47:01.287461 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.287426    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"]
Sep 11 04:47:01.290113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.290095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.292044 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.292024    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 04:47:01.292710 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.292691    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzm5d\""
Sep 11 04:47:01.292710 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.292701    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 04:47:01.298222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.298199    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"]
Sep 11 04:47:01.399302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.399268    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"]
Sep 11 04:47:01.401832 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.401815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.410312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.410284    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"]
Sep 11 04:47:01.430767 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.430736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.430767 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.430768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.430963 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.430818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94nr2\" (UniqueName: \"kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.490780 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.490750    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"]
Sep 11 04:47:01.493058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.493040    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.502152 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.502129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"]
Sep 11 04:47:01.531441 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.531611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq6rb\" (UniqueName: \"kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.531611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.531611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.531611 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.531749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94nr2\" (UniqueName: \"kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.531869 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.531904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.531857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.540541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.540482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94nr2\" (UniqueName: \"kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.599365 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.599328    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:01.606473 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.606448    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"]
Sep 11 04:47:01.609348 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.609193    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.623550 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.623523    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"]
Sep 11 04:47:01.632339 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.632440 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.632440 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.632518 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.632518 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6rb\" (UniqueName: \"kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.632624 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rppgj\" (UniqueName: \"kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.632779 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632758    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.632819 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.632777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.651119 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.650747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq6rb\" (UniqueName: \"kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.710985 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.710955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:01.733335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2lk\" (UniqueName: \"kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.733465 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.733524 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rppgj\" (UniqueName: \"kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.733737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.733823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.733823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.733796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.734157 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.734136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.734218 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.734192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.742212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.742181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rppgj\" (UniqueName: \"kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.802022 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.801948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:01.834828 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.834791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.834960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.834901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2lk\" (UniqueName: \"kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.834960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.834934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.835285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.835262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.835338 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.835286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.844022 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.843991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2lk\" (UniqueName: \"kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.876517 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.876488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"]
Sep 11 04:47:01.878610 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:47:01.878563    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165cd3fe_8ccd_4f6d_86fc_7233d757c868.slice/crio-0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa WatchSource:0}: Error finding container 0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa: Status 404 returned error can't find the container with id 0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa
Sep 11 04:47:01.929751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.929728    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:01.984935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:01.984825    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"]
Sep 11 04:47:02.025519 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:47:02.025486    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424f5935_1081_4ef3_8d32_e0d1712deac5.slice/crio-7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022 WatchSource:0}: Error finding container 7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022: Status 404 returned error can't find the container with id 7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022
Sep 11 04:47:02.063748 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.063715    2581 generic.go:356] "Generic (PLEG): container finished" podID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerID="f53639fca9ace6d2e6294abc70dc83761a453cd81565256c11b1585fcad1169f" exitCode=0
Sep 11 04:47:02.063895 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.063749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb" event={"ID":"165cd3fe-8ccd-4f6d-86fc-7233d757c868","Type":"ContainerDied","Data":"f53639fca9ace6d2e6294abc70dc83761a453cd81565256c11b1585fcad1169f"}
Sep 11 04:47:02.063895 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.063834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb" event={"ID":"165cd3fe-8ccd-4f6d-86fc-7233d757c868","Type":"ContainerStarted","Data":"0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa"}
Sep 11 04:47:02.065351 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.065328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz" event={"ID":"424f5935-1081-4ef3-8d32-e0d1712deac5","Type":"ContainerStarted","Data":"7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022"}
Sep 11 04:47:02.072343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.072323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"]
Sep 11 04:47:02.074566 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:47:02.074542    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cce11d_a382_48e1_b063_eb814908b209.slice/crio-a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6 WatchSource:0}: Error finding container a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6: Status 404 returned error can't find the container with id a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6
Sep 11 04:47:02.203541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:02.203506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"]
Sep 11 04:47:02.242446 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:47:02.242411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b81db8d_9e7b_40ff_a212_7a887aeebdb4.slice/crio-43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162 WatchSource:0}: Error finding container 43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162: Status 404 returned error can't find the container with id 43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162
Sep 11 04:47:03.071788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.071699    2581 generic.go:356] "Generic (PLEG): container finished" podID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerID="cc542996630fcbee3751546ca74e31b7e298bf9ede52616a2df4f6c5039ba492" exitCode=0
Sep 11 04:47:03.072237 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.071798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb" event={"ID":"165cd3fe-8ccd-4f6d-86fc-7233d757c868","Type":"ContainerDied","Data":"cc542996630fcbee3751546ca74e31b7e298bf9ede52616a2df4f6c5039ba492"}
Sep 11 04:47:03.073530 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.073504    2581 generic.go:356] "Generic (PLEG): container finished" podID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerID="2be302f9367c3a149170f161a20ca07bb771331cf8461aff6ec292d29acab0e8" exitCode=0
Sep 11 04:47:03.073713 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.073598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz" event={"ID":"424f5935-1081-4ef3-8d32-e0d1712deac5","Type":"ContainerDied","Data":"2be302f9367c3a149170f161a20ca07bb771331cf8461aff6ec292d29acab0e8"}
Sep 11 04:47:03.075192 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.075174    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerID="d58bd0e428da05f83aa0c4dd6c7cb01bb55510971aacc1bda81322a59a23668d" exitCode=0
Sep 11 04:47:03.075286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.075246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7" event={"ID":"1b81db8d-9e7b-40ff-a212-7a887aeebdb4","Type":"ContainerDied","Data":"d58bd0e428da05f83aa0c4dd6c7cb01bb55510971aacc1bda81322a59a23668d"}
Sep 11 04:47:03.075286 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.075280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7" event={"ID":"1b81db8d-9e7b-40ff-a212-7a887aeebdb4","Type":"ContainerStarted","Data":"43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162"}
Sep 11 04:47:03.076973 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.076952    2581 generic.go:356] "Generic (PLEG): container finished" podID="b5cce11d-a382-48e1-b063-eb814908b209" containerID="ff8889b53d6e81e6270452fe77300c62dde61f7535a6450e09c94d6cedf1fc29" exitCode=0
Sep 11 04:47:03.077051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.076981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4" event={"ID":"b5cce11d-a382-48e1-b063-eb814908b209","Type":"ContainerDied","Data":"ff8889b53d6e81e6270452fe77300c62dde61f7535a6450e09c94d6cedf1fc29"}
Sep 11 04:47:03.077051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:03.077010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4" event={"ID":"b5cce11d-a382-48e1-b063-eb814908b209","Type":"ContainerStarted","Data":"a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6"}
Sep 11 04:47:04.083908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.083873    2581 generic.go:356] "Generic (PLEG): container finished" podID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerID="675ef6a9f0f58e5b2af95a273c12cb8eb8d40a68544135f52cd68247a0a84fa9" exitCode=0
Sep 11 04:47:04.084355 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.083944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz" event={"ID":"424f5935-1081-4ef3-8d32-e0d1712deac5","Type":"ContainerDied","Data":"675ef6a9f0f58e5b2af95a273c12cb8eb8d40a68544135f52cd68247a0a84fa9"}
Sep 11 04:47:04.085857 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.085837    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerID="d131f848da69460294fd3195acd1c17709c31a88635889451db6255663e061a9" exitCode=0
Sep 11 04:47:04.085959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.085917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7" event={"ID":"1b81db8d-9e7b-40ff-a212-7a887aeebdb4","Type":"ContainerDied","Data":"d131f848da69460294fd3195acd1c17709c31a88635889451db6255663e061a9"}
Sep 11 04:47:04.088185 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.088164    2581 generic.go:356] "Generic (PLEG): container finished" podID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerID="8806679a4af4a2e9eede40ab4fe72c7df3a6c039820f9e61f821d70d901b0b62" exitCode=0
Sep 11 04:47:04.088269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:04.088229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb" event={"ID":"165cd3fe-8ccd-4f6d-86fc-7233d757c868","Type":"ContainerDied","Data":"8806679a4af4a2e9eede40ab4fe72c7df3a6c039820f9e61f821d70d901b0b62"}
Sep 11 04:47:05.096332 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.096248    2581 generic.go:356] "Generic (PLEG): container finished" podID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerID="6a10a7027de1999d4f5501cf4c141c978342dd926b02ba3d0bc15668979f7fb4" exitCode=0
Sep 11 04:47:05.096753 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.096341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz" event={"ID":"424f5935-1081-4ef3-8d32-e0d1712deac5","Type":"ContainerDied","Data":"6a10a7027de1999d4f5501cf4c141c978342dd926b02ba3d0bc15668979f7fb4"}
Sep 11 04:47:05.098477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.098455    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerID="911209afbcee1e199b581048d6fa6d799f0c39be1845718d7d96adaeac9c1efb" exitCode=0
Sep 11 04:47:05.098600 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.098485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7" event={"ID":"1b81db8d-9e7b-40ff-a212-7a887aeebdb4","Type":"ContainerDied","Data":"911209afbcee1e199b581048d6fa6d799f0c39be1845718d7d96adaeac9c1efb"}
Sep 11 04:47:05.100293 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.100276    2581 generic.go:356] "Generic (PLEG): container finished" podID="b5cce11d-a382-48e1-b063-eb814908b209" containerID="5102cb6504223355bfe24dafb6f5b84231436500094da0a0b21201d5c6f09e07" exitCode=0
Sep 11 04:47:05.100374 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.100359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4" event={"ID":"b5cce11d-a382-48e1-b063-eb814908b209","Type":"ContainerDied","Data":"5102cb6504223355bfe24dafb6f5b84231436500094da0a0b21201d5c6f09e07"}
Sep 11 04:47:05.233699 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.233674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:05.364934 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.364853    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94nr2\" (UniqueName: \"kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2\") pod \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") "
Sep 11 04:47:05.364934 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.364888    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util\") pod \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") "
Sep 11 04:47:05.364934 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.364932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle\") pod \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\" (UID: \"165cd3fe-8ccd-4f6d-86fc-7233d757c868\") "
Sep 11 04:47:05.365491 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.365466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle" (OuterVolumeSpecName: "bundle") pod "165cd3fe-8ccd-4f6d-86fc-7233d757c868" (UID: "165cd3fe-8ccd-4f6d-86fc-7233d757c868"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:05.367038 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.367015    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2" (OuterVolumeSpecName: "kube-api-access-94nr2") pod "165cd3fe-8ccd-4f6d-86fc-7233d757c868" (UID: "165cd3fe-8ccd-4f6d-86fc-7233d757c868"). InnerVolumeSpecName "kube-api-access-94nr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:47:05.370239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.370219    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util" (OuterVolumeSpecName: "util") pod "165cd3fe-8ccd-4f6d-86fc-7233d757c868" (UID: "165cd3fe-8ccd-4f6d-86fc-7233d757c868"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:05.466188 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.466163    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94nr2\" (UniqueName: \"kubernetes.io/projected/165cd3fe-8ccd-4f6d-86fc-7233d757c868-kube-api-access-94nr2\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:05.466188 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.466186    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:05.466357 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:05.466197    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/165cd3fe-8ccd-4f6d-86fc-7233d757c868-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.108633 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.108528    2581 generic.go:356] "Generic (PLEG): container finished" podID="b5cce11d-a382-48e1-b063-eb814908b209" containerID="7e3354cdfdd308a2b19a74dda702dbaec402db573b9241ecd5ac35f86c62b30a" exitCode=0
Sep 11 04:47:06.108633 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.108615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4" event={"ID":"b5cce11d-a382-48e1-b063-eb814908b209","Type":"ContainerDied","Data":"7e3354cdfdd308a2b19a74dda702dbaec402db573b9241ecd5ac35f86c62b30a"}
Sep 11 04:47:06.110476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.110458    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb"
Sep 11 04:47:06.110476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.110469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbk5tb" event={"ID":"165cd3fe-8ccd-4f6d-86fc-7233d757c868","Type":"ContainerDied","Data":"0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa"}
Sep 11 04:47:06.110653 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.110510    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f825ea074741f39149d6aea490c58acd3bafa7b2943bf33e40ba6a184034ffa"
Sep 11 04:47:06.263962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.263940    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:06.265939 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.265920    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:06.374034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.373951    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle\") pod \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") "
Sep 11 04:47:06.374034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.373996    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle\") pod \"424f5935-1081-4ef3-8d32-e0d1712deac5\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") "
Sep 11 04:47:06.374034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374012    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util\") pod \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") "
Sep 11 04:47:06.374283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util\") pod \"424f5935-1081-4ef3-8d32-e0d1712deac5\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") "
Sep 11 04:47:06.374283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq6rb\" (UniqueName: \"kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb\") pod \"424f5935-1081-4ef3-8d32-e0d1712deac5\" (UID: \"424f5935-1081-4ef3-8d32-e0d1712deac5\") "
Sep 11 04:47:06.374283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw2lk\" (UniqueName: \"kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk\") pod \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\" (UID: \"1b81db8d-9e7b-40ff-a212-7a887aeebdb4\") "
Sep 11 04:47:06.374565 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle" (OuterVolumeSpecName: "bundle") pod "424f5935-1081-4ef3-8d32-e0d1712deac5" (UID: "424f5935-1081-4ef3-8d32-e0d1712deac5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:06.374699 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.374539    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle" (OuterVolumeSpecName: "bundle") pod "1b81db8d-9e7b-40ff-a212-7a887aeebdb4" (UID: "1b81db8d-9e7b-40ff-a212-7a887aeebdb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:06.376435 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.376397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk" (OuterVolumeSpecName: "kube-api-access-sw2lk") pod "1b81db8d-9e7b-40ff-a212-7a887aeebdb4" (UID: "1b81db8d-9e7b-40ff-a212-7a887aeebdb4"). InnerVolumeSpecName "kube-api-access-sw2lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:47:06.376762 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.376735    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb" (OuterVolumeSpecName: "kube-api-access-dq6rb") pod "424f5935-1081-4ef3-8d32-e0d1712deac5" (UID: "424f5935-1081-4ef3-8d32-e0d1712deac5"). InnerVolumeSpecName "kube-api-access-dq6rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:47:06.379680 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.379658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util" (OuterVolumeSpecName: "util") pod "424f5935-1081-4ef3-8d32-e0d1712deac5" (UID: "424f5935-1081-4ef3-8d32-e0d1712deac5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:06.379819 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.379802    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util" (OuterVolumeSpecName: "util") pod "1b81db8d-9e7b-40ff-a212-7a887aeebdb4" (UID: "1b81db8d-9e7b-40ff-a212-7a887aeebdb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:06.474791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474760    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.474791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474787    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq6rb\" (UniqueName: \"kubernetes.io/projected/424f5935-1081-4ef3-8d32-e0d1712deac5-kube-api-access-dq6rb\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.474926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474798    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw2lk\" (UniqueName: \"kubernetes.io/projected/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-kube-api-access-sw2lk\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.474926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474807    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.474926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474815    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/424f5935-1081-4ef3-8d32-e0d1712deac5-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:06.474926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:06.474822    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b81db8d-9e7b-40ff-a212-7a887aeebdb4-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:07.116862 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.116826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz" event={"ID":"424f5935-1081-4ef3-8d32-e0d1712deac5","Type":"ContainerDied","Data":"7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022"}
Sep 11 04:47:07.117308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.116869    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f2aff803aa3d9125b008ddc5c7e3c5026293341ebb6c4e576c51b682c87f022"
Sep 11 04:47:07.117308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.116835    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62tztz"
Sep 11 04:47:07.118638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.118611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7" event={"ID":"1b81db8d-9e7b-40ff-a212-7a887aeebdb4","Type":"ContainerDied","Data":"43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162"}
Sep 11 04:47:07.118638 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.118635    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x9dh7"
Sep 11 04:47:07.118759 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.118660    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43870e4000f654a251828a3ead6be2f6bf9eb967d6db79ff35a7bb74a042a162"
Sep 11 04:47:07.245363 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.245343    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:07.381915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.381843    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rppgj\" (UniqueName: \"kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj\") pod \"b5cce11d-a382-48e1-b063-eb814908b209\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") "
Sep 11 04:47:07.381915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.381882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle\") pod \"b5cce11d-a382-48e1-b063-eb814908b209\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") "
Sep 11 04:47:07.381915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.381910    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util\") pod \"b5cce11d-a382-48e1-b063-eb814908b209\" (UID: \"b5cce11d-a382-48e1-b063-eb814908b209\") "
Sep 11 04:47:07.382568 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.382491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle" (OuterVolumeSpecName: "bundle") pod "b5cce11d-a382-48e1-b063-eb814908b209" (UID: "b5cce11d-a382-48e1-b063-eb814908b209"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:07.384094 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.384069    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj" (OuterVolumeSpecName: "kube-api-access-rppgj") pod "b5cce11d-a382-48e1-b063-eb814908b209" (UID: "b5cce11d-a382-48e1-b063-eb814908b209"). InnerVolumeSpecName "kube-api-access-rppgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:47:07.387270 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.387239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util" (OuterVolumeSpecName: "util") pod "b5cce11d-a382-48e1-b063-eb814908b209" (UID: "b5cce11d-a382-48e1-b063-eb814908b209"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:47:07.483475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.483448    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rppgj\" (UniqueName: \"kubernetes.io/projected/b5cce11d-a382-48e1-b063-eb814908b209-kube-api-access-rppgj\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:07.483475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.483470    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:07.483475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:07.483480    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5cce11d-a382-48e1-b063-eb814908b209-util\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:08.124802 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:08.124767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4" event={"ID":"b5cce11d-a382-48e1-b063-eb814908b209","Type":"ContainerDied","Data":"a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6"}
Sep 11 04:47:08.125169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:08.124811    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a327c7cf2f83b26813c7ff1709a7498be4ae94c030424e76cea733197590a0d6"
Sep 11 04:47:08.125169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:08.124775    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a8m5t4"
Sep 11 04:47:26.729437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:26.729400    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:47:51.754219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:51.754176    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5544b7bd74-k9lzq" podUID="3b757413-0a92-442d-8b0a-44735da25aa7" containerName="console" containerID="cri-o://1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046" gracePeriod=15
Sep 11 04:47:51.988869 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:51.988846    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:47:52.029452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029386    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf7mz\" (UniqueName: \"kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029418    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029497    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029552    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029610    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029637    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config\") pod \"3b757413-0a92-442d-8b0a-44735da25aa7\" (UID: \"3b757413-0a92-442d-8b0a-44735da25aa7\") "
Sep 11 04:47:52.029949 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029909    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:47:52.030006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.029933    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca" (OuterVolumeSpecName: "service-ca") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:47:52.030063 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.030042    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:47:52.030119 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.030105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config" (OuterVolumeSpecName: "console-config") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:47:52.031715 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.031694    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:47:52.031715 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.031706    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz" (OuterVolumeSpecName: "kube-api-access-rf7mz") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "kube-api-access-rf7mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:47:52.031842 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.031742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3b757413-0a92-442d-8b0a-44735da25aa7" (UID: "3b757413-0a92-442d-8b0a-44735da25aa7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:47:52.130615 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130564    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-serving-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130615 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130609    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-oauth-serving-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130615 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130620    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-console-config\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130628    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf7mz\" (UniqueName: \"kubernetes.io/projected/3b757413-0a92-442d-8b0a-44735da25aa7-kube-api-access-rf7mz\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130637    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-trusted-ca-bundle\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130645    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b757413-0a92-442d-8b0a-44735da25aa7-console-oauth-config\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.130830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.130654    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b757413-0a92-442d-8b0a-44735da25aa7-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:47:52.332069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.331998    2581 generic.go:356] "Generic (PLEG): container finished" podID="3b757413-0a92-442d-8b0a-44735da25aa7" containerID="1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046" exitCode=0
Sep 11 04:47:52.332069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.332042    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5544b7bd74-k9lzq"
Sep 11 04:47:52.332267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.332087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5544b7bd74-k9lzq" event={"ID":"3b757413-0a92-442d-8b0a-44735da25aa7","Type":"ContainerDied","Data":"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"}
Sep 11 04:47:52.332267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.332141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5544b7bd74-k9lzq" event={"ID":"3b757413-0a92-442d-8b0a-44735da25aa7","Type":"ContainerDied","Data":"3709a9724512c3a9e700b13d400a370aa64dc1f9dec37b531e768a9d8520d6c0"}
Sep 11 04:47:52.332267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.332169    2581 scope.go:117] "RemoveContainer" containerID="1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"
Sep 11 04:47:52.340518 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.340501    2581 scope.go:117] "RemoveContainer" containerID="1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"
Sep 11 04:47:52.340798 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:47:52.340779    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046\": container with ID starting with 1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046 not found: ID does not exist" containerID="1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"
Sep 11 04:47:52.340851 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.340807    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046"} err="failed to get container status \"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046\": rpc error: code = NotFound desc = could not find container \"1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046\": container with ID starting with 1bb52b04cd6f972e2adfc155fea02537998394e2d8cc242b9e9807218dd90046 not found: ID does not exist"
Sep 11 04:47:52.347244 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.347224    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:47:52.353423 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:52.353403    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5544b7bd74-k9lzq"]
Sep 11 04:47:54.193558 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:47:54.193528    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b757413-0a92-442d-8b0a-44735da25aa7" path="/var/lib/kubelet/pods/3b757413-0a92-442d-8b0a-44735da25aa7/volumes"
Sep 11 04:48:12.831641 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831605    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831874    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831884    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831892    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831898    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831906    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831911    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831917    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831922    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831931    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831936    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831943    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b757413-0a92-442d-8b0a-44735da25aa7" containerName="console"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831948    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b757413-0a92-442d-8b0a-44735da25aa7" containerName="console"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831955    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831960    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831966    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831971    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831975    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831980    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="util"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831984    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.831989    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="extract"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832001    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="pull"
Sep 11 04:48:12.831995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832006    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="pull"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832011    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832015    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832021    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="util"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832025    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="util"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832071    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5cce11d-a382-48e1-b063-eb814908b209" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832079    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="424f5935-1081-4ef3-8d32-e0d1712deac5" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832085    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="165cd3fe-8ccd-4f6d-86fc-7233d757c868" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832091    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b81db8d-9e7b-40ff-a212-7a887aeebdb4" containerName="extract"
Sep 11 04:48:12.832657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.832097    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b757413-0a92-442d-8b0a-44735da25aa7" containerName="console"
Sep 11 04:48:12.833985 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.833970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:12.836006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.835989    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 04:48:12.836654 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.836632    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 04:48:12.836770 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.836697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6zshq\""
Sep 11 04:48:12.842288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.842268    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:12.992162 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:12.992131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brzpw\" (UniqueName: \"kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw\") pod \"authorino-7bcfd8f9cd-52ng9\" (UID: \"41286595-4ee7-41df-b91e-7a4a26b4b61e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:13.092736 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:13.092658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brzpw\" (UniqueName: \"kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw\") pod \"authorino-7bcfd8f9cd-52ng9\" (UID: \"41286595-4ee7-41df-b91e-7a4a26b4b61e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:13.101231 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:13.101198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brzpw\" (UniqueName: \"kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw\") pod \"authorino-7bcfd8f9cd-52ng9\" (UID: \"41286595-4ee7-41df-b91e-7a4a26b4b61e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:13.144269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:13.144243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:13.409273 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:13.409247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:13.411294 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:48:13.411264    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41286595_4ee7_41df_b91e_7a4a26b4b61e.slice/crio-43f2aec5498cd97246d03e15a000f77c2fe73119a7eca305106190cf1044c9f9 WatchSource:0}: Error finding container 43f2aec5498cd97246d03e15a000f77c2fe73119a7eca305106190cf1044c9f9: Status 404 returned error can't find the container with id 43f2aec5498cd97246d03e15a000f77c2fe73119a7eca305106190cf1044c9f9
Sep 11 04:48:13.430084 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:13.430054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" event={"ID":"41286595-4ee7-41df-b91e-7a4a26b4b61e","Type":"ContainerStarted","Data":"43f2aec5498cd97246d03e15a000f77c2fe73119a7eca305106190cf1044c9f9"}
Sep 11 04:48:15.997237 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:15.997197    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:16.448094 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:16.448058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" event={"ID":"41286595-4ee7-41df-b91e-7a4a26b4b61e","Type":"ContainerStarted","Data":"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"}
Sep 11 04:48:16.467290 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:16.467231    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" podStartSLOduration=2.043640704 podStartE2EDuration="4.467212681s" totalImagesPullingTime="2.423571977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:48:16.466921195 +0000 UTC m=+490.758395632" watchObservedRunningTime="2026-09-11 04:48:16.467212681 +0000 UTC m=+490.758687136"
Sep 11 04:48:17.452404 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:17.452362    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" podUID="41286595-4ee7-41df-b91e-7a4a26b4b61e" containerName="authorino" containerID="cri-o://813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358" gracePeriod=30
Sep 11 04:48:17.688942 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:17.688921    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:17.835629 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:17.835514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brzpw\" (UniqueName: \"kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw\") pod \"41286595-4ee7-41df-b91e-7a4a26b4b61e\" (UID: \"41286595-4ee7-41df-b91e-7a4a26b4b61e\") "
Sep 11 04:48:17.837761 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:17.837732    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw" (OuterVolumeSpecName: "kube-api-access-brzpw") pod "41286595-4ee7-41df-b91e-7a4a26b4b61e" (UID: "41286595-4ee7-41df-b91e-7a4a26b4b61e"). InnerVolumeSpecName "kube-api-access-brzpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:48:17.937120 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:17.937071    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brzpw\" (UniqueName: \"kubernetes.io/projected/41286595-4ee7-41df-b91e-7a4a26b4b61e-kube-api-access-brzpw\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:48:18.458230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.458142    2581 generic.go:356] "Generic (PLEG): container finished" podID="41286595-4ee7-41df-b91e-7a4a26b4b61e" containerID="813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358" exitCode=0
Sep 11 04:48:18.458230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.458188    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9"
Sep 11 04:48:18.458711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.458227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" event={"ID":"41286595-4ee7-41df-b91e-7a4a26b4b61e","Type":"ContainerDied","Data":"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"}
Sep 11 04:48:18.458711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.458274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ng9" event={"ID":"41286595-4ee7-41df-b91e-7a4a26b4b61e","Type":"ContainerDied","Data":"43f2aec5498cd97246d03e15a000f77c2fe73119a7eca305106190cf1044c9f9"}
Sep 11 04:48:18.458711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.458292    2581 scope.go:117] "RemoveContainer" containerID="813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"
Sep 11 04:48:18.467020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.467002    2581 scope.go:117] "RemoveContainer" containerID="813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"
Sep 11 04:48:18.467285 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:48:18.467267    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358\": container with ID starting with 813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358 not found: ID does not exist" containerID="813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"
Sep 11 04:48:18.467330 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.467292    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358"} err="failed to get container status \"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358\": rpc error: code = NotFound desc = could not find container \"813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358\": container with ID starting with 813e668b02843761c599a7f2a61e02ce66563dc4ad4e2cf60b22081035449358 not found: ID does not exist"
Sep 11 04:48:18.472718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.472697    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:18.477595 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:18.477561    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ng9"]
Sep 11 04:48:20.191861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:20.191832    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41286595-4ee7-41df-b91e-7a4a26b4b61e" path="/var/lib/kubelet/pods/41286595-4ee7-41df-b91e-7a4a26b4b61e/volumes"
Sep 11 04:48:41.237502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.237467    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-844666df4b-nhjxh"]
Sep 11 04:48:41.237993 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.237760    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41286595-4ee7-41df-b91e-7a4a26b4b61e" containerName="authorino"
Sep 11 04:48:41.237993 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.237771    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="41286595-4ee7-41df-b91e-7a4a26b4b61e" containerName="authorino"
Sep 11 04:48:41.237993 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.237846    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="41286595-4ee7-41df-b91e-7a4a26b4b61e" containerName="authorino"
Sep 11 04:48:41.240714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.240700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.242636 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.242610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 04:48:41.243212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.243198    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6zshq\""
Sep 11 04:48:41.243291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.243212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 04:48:41.247078 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.247057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-844666df4b-nhjxh"]
Sep 11 04:48:41.288428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.288396    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-844666df4b-nhjxh"]
Sep 11 04:48:41.288655 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:48:41.288637    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-275m9], unattached volumes=[], failed to process volumes=[kube-api-access-275m9]: context canceled" pod="kuadrant-system/authorino-844666df4b-nhjxh" podUID="0f929b71-aca7-4516-8ea4-db85845f9881"
Sep 11 04:48:41.312991 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.312959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-275m9\" (UniqueName: \"kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9\") pod \"authorino-844666df4b-nhjxh\" (UID: \"0f929b71-aca7-4516-8ea4-db85845f9881\") " pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.314732 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.314711    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:48:41.317767 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.317753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.319865 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.319843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 04:48:41.324693 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.324676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:48:41.414343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.414315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdlng\" (UniqueName: \"kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.414493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.414362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-275m9\" (UniqueName: \"kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9\") pod \"authorino-844666df4b-nhjxh\" (UID: \"0f929b71-aca7-4516-8ea4-db85845f9881\") " pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.414493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.414380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.422763 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.422735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-275m9\" (UniqueName: \"kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9\") pod \"authorino-844666df4b-nhjxh\" (UID: \"0f929b71-aca7-4516-8ea4-db85845f9881\") " pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.515973 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.515898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdlng\" (UniqueName: \"kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.515973 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.515961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.518470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.518452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.523847 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.523827    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdlng\" (UniqueName: \"kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng\") pod \"authorino-54b47d8f9b-x5rm6\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") " pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.565978 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.565947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.572889 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.572871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:41.616700 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.616674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-275m9\" (UniqueName: \"kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9\") pod \"0f929b71-aca7-4516-8ea4-db85845f9881\" (UID: \"0f929b71-aca7-4516-8ea4-db85845f9881\") "
Sep 11 04:48:41.618837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.618811    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9" (OuterVolumeSpecName: "kube-api-access-275m9") pod "0f929b71-aca7-4516-8ea4-db85845f9881" (UID: "0f929b71-aca7-4516-8ea4-db85845f9881"). InnerVolumeSpecName "kube-api-access-275m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:48:41.626951 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.626936    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:48:41.717926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.717895    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-275m9\" (UniqueName: \"kubernetes.io/projected/0f929b71-aca7-4516-8ea4-db85845f9881-kube-api-access-275m9\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:48:41.896720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:41.896671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:48:41.899254 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:48:41.899227    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecf8c57b_bfa7_428c_b06a_d097dbbf0cca.slice/crio-4674442fe71d9bbaaef87db91b2ddb32031c894160fae0888b6745991e4f93f3 WatchSource:0}: Error finding container 4674442fe71d9bbaaef87db91b2ddb32031c894160fae0888b6745991e4f93f3: Status 404 returned error can't find the container with id 4674442fe71d9bbaaef87db91b2ddb32031c894160fae0888b6745991e4f93f3
Sep 11 04:48:42.572734 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.572695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" event={"ID":"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca","Type":"ContainerStarted","Data":"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"}
Sep 11 04:48:42.573102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.572748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" event={"ID":"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca","Type":"ContainerStarted","Data":"4674442fe71d9bbaaef87db91b2ddb32031c894160fae0888b6745991e4f93f3"}
Sep 11 04:48:42.573102 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.572792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-844666df4b-nhjxh"
Sep 11 04:48:42.586122 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.586075    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" podStartSLOduration=1.079820273 podStartE2EDuration="1.586062273s" totalImagesPullingTime="506.242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:48:42.585115651 +0000 UTC m=+516.876590088" watchObservedRunningTime="2026-09-11 04:48:42.586062273 +0000 UTC m=+516.877536710"
Sep 11 04:48:42.611678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.611650    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-844666df4b-nhjxh"]
Sep 11 04:48:42.618626 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:42.618602    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-844666df4b-nhjxh"]
Sep 11 04:48:44.192491 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:48:44.192460    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f929b71-aca7-4516-8ea4-db85845f9881" path="/var/lib/kubelet/pods/0f929b71-aca7-4516-8ea4-db85845f9881/volumes"
Sep 11 04:49:52.947015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.946979    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"]
Sep 11 04:49:52.950403 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.950379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.952656 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.952634    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 04:49:52.953616 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.953597    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 04:49:52.953616 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.953614    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tzcgp\""
Sep 11 04:49:52.953777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.953634    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 04:49:52.964728 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.964706    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"]
Sep 11 04:49:52.995088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtwt4\" (UniqueName: \"kubernetes.io/projected/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-kube-api-access-gtwt4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.995200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.995200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.995200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.995305 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:52.995305 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:52.995267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095738 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095738 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtwt4\" (UniqueName: \"kubernetes.io/projected/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-kube-api-access-gtwt4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.095935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.095893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.096192 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.096170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.096244 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.096195    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.096299 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.096271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.098153 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.098133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.098403 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.098387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.106326 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.106304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtwt4\" (UniqueName: \"kubernetes.io/projected/9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3-kube-api-access-gtwt4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsjfp\" (UID: \"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.260944 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.260870    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:49:53.531561 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.531540    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"]
Sep 11 04:49:53.533556 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:49:53.533530    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccc911f_eaf0_4917_80d4_ae7a2fbf27a3.slice/crio-6d4a16fde6f7ddd020dd442a4aa96cc557dc9d01416a4c8d22219603dd1b1aec WatchSource:0}: Error finding container 6d4a16fde6f7ddd020dd442a4aa96cc557dc9d01416a4c8d22219603dd1b1aec: Status 404 returned error can't find the container with id 6d4a16fde6f7ddd020dd442a4aa96cc557dc9d01416a4c8d22219603dd1b1aec
Sep 11 04:49:53.912657 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:53.912535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp" event={"ID":"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3","Type":"ContainerStarted","Data":"6d4a16fde6f7ddd020dd442a4aa96cc557dc9d01416a4c8d22219603dd1b1aec"}
Sep 11 04:49:54.920945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:54.920868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp" event={"ID":"9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3","Type":"ContainerStarted","Data":"c563683a39158f2b443ab462ebd670808fa26478298eee0c99feb4a870a4dfb5"}
Sep 11 04:49:54.940484 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:49:54.940432    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp" podStartSLOduration=1.8910463530000001 podStartE2EDuration="2.940417583s" totalImagesPullingTime="1.04937123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:49:54.939301072 +0000 UTC m=+589.230775509" watchObservedRunningTime="2026-09-11 04:49:54.940417583 +0000 UTC m=+589.231892032"
Sep 11 04:50:03.262019 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:03.261982    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:50:03.262442 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:03.262078    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:50:03.274664 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:03.274632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:50:03.977721 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:03.977685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsjfp"
Sep 11 04:50:06.127180 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:06.127147    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:50:06.127642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:06.127603    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:50:16.348555 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.348524    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"]
Sep 11 04:50:16.351721 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.351702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.353785 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.353766    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 04:50:16.359311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.359293    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"]
Sep 11 04:50:16.490933 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.490901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.490933 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.490938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f618774-b247-4fa7-888a-1f72eed8f3f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.491133 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.490956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.491133 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.490973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.491133 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.491018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbj9f\" (UniqueName: \"kubernetes.io/projected/5f618774-b247-4fa7-888a-1f72eed8f3f9-kube-api-access-kbj9f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.491133 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.491062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.591843 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f618774-b247-4fa7-888a-1f72eed8f3f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.591993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbj9f\" (UniqueName: \"kubernetes.io/projected/5f618774-b247-4fa7-888a-1f72eed8f3f9-kube-api-access-kbj9f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592280 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.592260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.592313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.592408 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.592381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.594375 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.594354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f618774-b247-4fa7-888a-1f72eed8f3f9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.594664 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.594648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f618774-b247-4fa7-888a-1f72eed8f3f9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.599612 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.599546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbj9f\" (UniqueName: \"kubernetes.io/projected/5f618774-b247-4fa7-888a-1f72eed8f3f9-kube-api-access-kbj9f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f\" (UID: \"5f618774-b247-4fa7-888a-1f72eed8f3f9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.665765 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.665738    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:16.935404 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.935378    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"]
Sep 11 04:50:16.937264 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:50:16.937237    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f618774_b247_4fa7_888a_1f72eed8f3f9.slice/crio-84fa6fe7ece927c9dbd13140f3628af979bc030e3094d188bc8fec83ba291094 WatchSource:0}: Error finding container 84fa6fe7ece927c9dbd13140f3628af979bc030e3094d188bc8fec83ba291094: Status 404 returned error can't find the container with id 84fa6fe7ece927c9dbd13140f3628af979bc030e3094d188bc8fec83ba291094
Sep 11 04:50:16.938533 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:16.938516    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:50:17.033808 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:17.033762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f" event={"ID":"5f618774-b247-4fa7-888a-1f72eed8f3f9","Type":"ContainerStarted","Data":"84fa6fe7ece927c9dbd13140f3628af979bc030e3094d188bc8fec83ba291094"}
Sep 11 04:50:18.041386 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:18.041323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f" event={"ID":"5f618774-b247-4fa7-888a-1f72eed8f3f9","Type":"ContainerStarted","Data":"1ddffe160da9177e4f70d73f1c2b79b8ab74f9ed5ea17214e8711208a1736601"}
Sep 11 04:50:18.059069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:18.059014    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f" podStartSLOduration=1.858405622 podStartE2EDuration="2.058998117s" totalImagesPullingTime="200.592495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:50:18.058377878 +0000 UTC m=+612.349852316" watchObservedRunningTime="2026-09-11 04:50:18.058998117 +0000 UTC m=+612.350472566"
Sep 11 04:50:26.665914 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:26.665875    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:26.666359 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:26.665951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:26.678338 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:26.678311    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:27.097051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:27.097024    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f"
Sep 11 04:50:29.447872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.447789    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"]
Sep 11 04:50:29.451839 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.451792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.454553 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.454535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 04:50:29.462970 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.462953    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"]
Sep 11 04:50:29.594139 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.594321 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.594321 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.594321 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.594321 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czt9p\" (UniqueName: \"kubernetes.io/projected/722d726a-fe13-4a9c-bdab-77d73891ec2d-kube-api-access-czt9p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.594488 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.594345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/722d726a-fe13-4a9c-bdab-77d73891ec2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695416 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czt9p\" (UniqueName: \"kubernetes.io/projected/722d726a-fe13-4a9c-bdab-77d73891ec2d-kube-api-access-czt9p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/722d726a-fe13-4a9c-bdab-77d73891ec2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695800 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695860 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.695999 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.695957    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.697967 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.697923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/722d726a-fe13-4a9c-bdab-77d73891ec2d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.698188 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.698172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/722d726a-fe13-4a9c-bdab-77d73891ec2d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.703535 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.703517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czt9p\" (UniqueName: \"kubernetes.io/projected/722d726a-fe13-4a9c-bdab-77d73891ec2d-kube-api-access-czt9p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hp5nn\" (UID: \"722d726a-fe13-4a9c-bdab-77d73891ec2d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:29.762041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:29.762015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:30.032639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:30.032614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"]
Sep 11 04:50:30.034252 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:50:30.034226    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod722d726a_fe13_4a9c_bdab_77d73891ec2d.slice/crio-0d14cd634cd14b81651f1707f1277cb310c599dcc2e3059dfce00963ed3205e3 WatchSource:0}: Error finding container 0d14cd634cd14b81651f1707f1277cb310c599dcc2e3059dfce00963ed3205e3: Status 404 returned error can't find the container with id 0d14cd634cd14b81651f1707f1277cb310c599dcc2e3059dfce00963ed3205e3
Sep 11 04:50:30.101251 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:30.101216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn" event={"ID":"722d726a-fe13-4a9c-bdab-77d73891ec2d","Type":"ContainerStarted","Data":"0d14cd634cd14b81651f1707f1277cb310c599dcc2e3059dfce00963ed3205e3"}
Sep 11 04:50:31.107367 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:31.107326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn" event={"ID":"722d726a-fe13-4a9c-bdab-77d73891ec2d","Type":"ContainerStarted","Data":"de8d9a0ad809f48af9fccfe96ac9746f6bfc0d6c79ea5d14ef7580f7a6967a88"}
Sep 11 04:50:31.127187 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:31.127138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn" podStartSLOduration=1.954371655 podStartE2EDuration="2.127122173s" totalImagesPullingTime="172.750518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:50:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:50:31.12449019 +0000 UTC m=+625.415964627" watchObservedRunningTime="2026-09-11 04:50:31.127122173 +0000 UTC m=+625.418596610"
Sep 11 04:50:39.762214 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:39.762174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:39.762637 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:39.762469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:39.774471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:39.774447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:40.053266 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.053186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"]
Sep 11 04:50:40.056481 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.056464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.058609 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.058575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 04:50:40.066302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.066280    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"]
Sep 11 04:50:40.082212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.082328 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.082328 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.082328 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.082478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lpjz\" (UniqueName: \"kubernetes.io/projected/423573b6-4e7d-4986-87ef-d0224ee401be-kube-api-access-2lpjz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.082478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.082437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/423573b6-4e7d-4986-87ef-d0224ee401be-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.160603 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.160564    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hp5nn"
Sep 11 04:50:40.183212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lpjz\" (UniqueName: \"kubernetes.io/projected/423573b6-4e7d-4986-87ef-d0224ee401be-kube-api-access-2lpjz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/423573b6-4e7d-4986-87ef-d0224ee401be-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183666 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.183835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.183731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.185923 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.185898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/423573b6-4e7d-4986-87ef-d0224ee401be-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.186090 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.186071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/423573b6-4e7d-4986-87ef-d0224ee401be-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.191993 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.191967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lpjz\" (UniqueName: \"kubernetes.io/projected/423573b6-4e7d-4986-87ef-d0224ee401be-kube-api-access-2lpjz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz\" (UID: \"423573b6-4e7d-4986-87ef-d0224ee401be\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.367390 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.367309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:40.641157 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:40.641128    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"]
Sep 11 04:50:40.643209 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:50:40.643187    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod423573b6_4e7d_4986_87ef_d0224ee401be.slice/crio-5a35fb84e01d8bb17cca47e757a46f462cfc106cd8c92edd061130db6012004f WatchSource:0}: Error finding container 5a35fb84e01d8bb17cca47e757a46f462cfc106cd8c92edd061130db6012004f: Status 404 returned error can't find the container with id 5a35fb84e01d8bb17cca47e757a46f462cfc106cd8c92edd061130db6012004f
Sep 11 04:50:41.155406 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:41.155367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz" event={"ID":"423573b6-4e7d-4986-87ef-d0224ee401be","Type":"ContainerStarted","Data":"100ddc4bc9132be67c417f334f75a0137360ab203868ec506db5cf888b772919"}
Sep 11 04:50:41.155406 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:41.155411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz" event={"ID":"423573b6-4e7d-4986-87ef-d0224ee401be","Type":"ContainerStarted","Data":"5a35fb84e01d8bb17cca47e757a46f462cfc106cd8c92edd061130db6012004f"}
Sep 11 04:50:41.172764 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:41.172699    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz" podStartSLOduration=0.980075786 podStartE2EDuration="1.172684835s" totalImagesPullingTime="192.609049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:50:41.170898769 +0000 UTC m=+635.462373207" watchObservedRunningTime="2026-09-11 04:50:41.172684835 +0000 UTC m=+635.464159271"
Sep 11 04:50:50.367886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:50.367830    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:50.367886 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:50.367895    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:50.380647 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:50.380619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:50:51.216839 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:50:51.216812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz"
Sep 11 04:52:09.220380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.220347    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:52:09.220899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.220525    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" podUID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" containerName="authorino" containerID="cri-o://9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214" gracePeriod=30
Sep 11 04:52:09.464511 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.464487    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:52:09.549990 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.549913    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdlng\" (UniqueName: \"kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng\") pod \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") "
Sep 11 04:52:09.550142 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.550002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert\") pod \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\" (UID: \"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca\") "
Sep 11 04:52:09.552096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.552072    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng" (OuterVolumeSpecName: "kube-api-access-wdlng") pod "ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" (UID: "ecf8c57b-bfa7-428c-b06a-d097dbbf0cca"). InnerVolumeSpecName "kube-api-access-wdlng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:52:09.561112 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.561092    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" (UID: "ecf8c57b-bfa7-428c-b06a-d097dbbf0cca"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:52:09.599689 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.599659    2581 generic.go:356] "Generic (PLEG): container finished" podID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" containerID="9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214" exitCode=0
Sep 11 04:52:09.599786 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.599703    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6"
Sep 11 04:52:09.599786 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.599722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" event={"ID":"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca","Type":"ContainerDied","Data":"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"}
Sep 11 04:52:09.599786 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.599753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54b47d8f9b-x5rm6" event={"ID":"ecf8c57b-bfa7-428c-b06a-d097dbbf0cca","Type":"ContainerDied","Data":"4674442fe71d9bbaaef87db91b2ddb32031c894160fae0888b6745991e4f93f3"}
Sep 11 04:52:09.599786 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.599768    2581 scope.go:117] "RemoveContainer" containerID="9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"
Sep 11 04:52:09.608908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.608891    2581 scope.go:117] "RemoveContainer" containerID="9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"
Sep 11 04:52:09.609148 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:52:09.609127    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214\": container with ID starting with 9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214 not found: ID does not exist" containerID="9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"
Sep 11 04:52:09.609213 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.609155    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214"} err="failed to get container status \"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214\": rpc error: code = NotFound desc = could not find container \"9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214\": container with ID starting with 9682ce18a658e8b5dd53fd0b2471f87caa2e9b10829b9649d4b891abdc085214 not found: ID does not exist"
Sep 11 04:52:09.618941 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.618856    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:52:09.621063 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.621041    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54b47d8f9b-x5rm6"]
Sep 11 04:52:09.650855 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.650824    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdlng\" (UniqueName: \"kubernetes.io/projected/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-kube-api-access-wdlng\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:52:09.650855 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:09.650852    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca-tls-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:52:10.198017 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:10.197987    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" path="/var/lib/kubelet/pods/ecf8c57b-bfa7-428c-b06a-d097dbbf0cca/volumes"
Sep 11 04:52:32.009119 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.009085    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7527-426d95"]
Sep 11 04:52:32.009525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.009421    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" containerName="authorino"
Sep 11 04:52:32.009525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.009434    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" containerName="authorino"
Sep 11 04:52:32.009525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.009494    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf8c57b-bfa7-428c-b06a-d097dbbf0cca" containerName="authorino"
Sep 11 04:52:32.012432 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.012416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:32.014683 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.014661    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mgwvk\""
Sep 11 04:52:32.016279 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.016255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7527-426d95"]
Sep 11 04:52:32.134499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.134472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7n2m\" (UniqueName: \"kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m\") pod \"test-curl-7527-426d95\" (UID: \"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098\") " pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:32.235073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.235047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7n2m\" (UniqueName: \"kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m\") pod \"test-curl-7527-426d95\" (UID: \"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098\") " pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:32.244148 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.244126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7n2m\" (UniqueName: \"kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m\") pod \"test-curl-7527-426d95\" (UID: \"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098\") " pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:32.323754 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.323705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:32.590678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.590653    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7527-426d95"]
Sep 11 04:52:32.592155 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:52:32.592129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5d9b698_ab23_4f86_9a4c_e74d2c9a9098.slice/crio-bc1916beccb942f30ca7e2121288256e2bafde48d95039e9973fffab5b735188 WatchSource:0}: Error finding container bc1916beccb942f30ca7e2121288256e2bafde48d95039e9973fffab5b735188: Status 404 returned error can't find the container with id bc1916beccb942f30ca7e2121288256e2bafde48d95039e9973fffab5b735188
Sep 11 04:52:32.718899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:32.718868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7527-426d95" event={"ID":"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098","Type":"ContainerStarted","Data":"bc1916beccb942f30ca7e2121288256e2bafde48d95039e9973fffab5b735188"}
Sep 11 04:52:33.725269 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:33.725238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7527-426d95" event={"ID":"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098","Type":"ContainerStarted","Data":"b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100"}
Sep 11 04:52:33.741347 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:33.741299    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7527-426d95" podStartSLOduration=1.788237993 podStartE2EDuration="2.741284656s" totalImagesPullingTime="953.046663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:33.739268973 +0000 UTC m=+748.030743411" watchObservedRunningTime="2026-09-11 04:52:33.741284656 +0000 UTC m=+748.032759092"
Sep 11 04:52:34.311518 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:34.311487    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7527-426d95"]
Sep 11 04:52:34.319996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:34.319976    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7527-426d95"]
Sep 11 04:52:34.730207 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:52:34.730181    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7527-426d95" container="test-curl-7527-426d95"
Sep 11 04:52:35.090919 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.090828    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:52:35.091241 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.091225    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" containerName="test-curl-7527-426d95"
Sep 11 04:52:35.091308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.091246    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" containerName="test-curl-7527-426d95"
Sep 11 04:52:35.091351 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.091308    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" containerName="test-curl-7527-426d95"
Sep 11 04:52:35.094372 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.094353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.096495 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.096476    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-0a8fef-openshift-default-dockercfg-4vlg6\""
Sep 11 04:52:35.105332 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.105310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:52:35.155791 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.155895 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.155895 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtcb\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.155895 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.156000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.156000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.156000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.156000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.156000 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.155988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.256899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.256899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.256984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtcb\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257480 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257480 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257480 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257683 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.257815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.257793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.259343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.259322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.259430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.259415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.264254 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.264227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.264477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.264460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtcb\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb\") pod \"e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") " pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.404907 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.404827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:35.678878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.678850    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:52:35.680001 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:52:35.679972    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e10ac2_efe3_4d00_ac05_87521e28de2b.slice/crio-9f38572d0d2c5e1ea41fe0a7bbb2a73b0a903887a07d5e3453945a7c9baed4a3 WatchSource:0}: Error finding container 9f38572d0d2c5e1ea41fe0a7bbb2a73b0a903887a07d5e3453945a7c9baed4a3: Status 404 returned error can't find the container with id 9f38572d0d2c5e1ea41fe0a7bbb2a73b0a903887a07d5e3453945a7c9baed4a3
Sep 11 04:52:35.682199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.682171    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:35.682267 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.682235    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:35.682301 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.682284    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:35.735911 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.735887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerStarted","Data":"9f38572d0d2c5e1ea41fe0a7bbb2a73b0a903887a07d5e3453945a7c9baed4a3"}
Sep 11 04:52:35.736202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:35.735966    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7527-426d95" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" containerName="test-curl-7527-426d95" containerID="cri-o://b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100" gracePeriod=2
Sep 11 04:52:35.737943 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:52:35.737912    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7527-426d95\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:36.191499 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:52:36.191454    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7527-426d95\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:36.742569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:36.742524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerStarted","Data":"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"}
Sep 11 04:52:36.768505 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:36.768454    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podStartSLOduration=1.768439301 podStartE2EDuration="1.768439301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:36.767502218 +0000 UTC m=+751.058976658" watchObservedRunningTime="2026-09-11 04:52:36.768439301 +0000 UTC m=+751.059913732"
Sep 11 04:52:37.405311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.405284    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:37.409574 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.409553    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:37.748393 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.748372    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:37.749349 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.749319    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:37.872671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.872650    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:37.978310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.978278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7n2m\" (UniqueName: \"kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m\") pod \"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098\" (UID: \"f5d9b698-ab23-4f86-9a4c-e74d2c9a9098\") "
Sep 11 04:52:37.980465 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:37.980439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m" (OuterVolumeSpecName: "kube-api-access-p7n2m") pod "f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" (UID: "f5d9b698-ab23-4f86-9a4c-e74d2c9a9098"). InnerVolumeSpecName "kube-api-access-p7n2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:52:38.078992 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.078908    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7n2m\" (UniqueName: \"kubernetes.io/projected/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098-kube-api-access-p7n2m\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:52:38.192283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.192253    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" path="/var/lib/kubelet/pods/f5d9b698-ab23-4f86-9a4c-e74d2c9a9098/volumes"
Sep 11 04:52:38.754528 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.754493    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5d9b698-ab23-4f86-9a4c-e74d2c9a9098" containerID="b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100" exitCode=137
Sep 11 04:52:38.755016 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.754538    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7527-426d95"
Sep 11 04:52:38.755016 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.754617    2581 scope.go:117] "RemoveContainer" containerID="b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100"
Sep 11 04:52:38.762929 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.762913    2581 scope.go:117] "RemoveContainer" containerID="b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100"
Sep 11 04:52:38.763168 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:52:38.763150    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100\": container with ID starting with b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100 not found: ID does not exist" containerID="b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100"
Sep 11 04:52:38.763230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:38.763181    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100"} err="failed to get container status \"b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100\": rpc error: code = NotFound desc = could not find container \"b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100\": container with ID starting with b99175b7321fe4819107fd262e646e0c247e4626f61e3d574bcb07ab3c012100 not found: ID does not exist"
Sep 11 04:52:39.045282 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.045213    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:52:39.049753 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.049738    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.051741 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.051720    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-a0a72050-kserve-self-signed-certs\""
Sep 11 04:52:39.062401 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.062370    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:52:39.194957 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.194924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.194957 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.194955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.195146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.194969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.195146 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.195043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.195248 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.195159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cwnv\" (UniqueName: \"kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.195248 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.195194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.295835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.295770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cwnv\" (UniqueName: \"kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.295835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.295804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.295835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.295825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296058 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296166 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.296389 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.296373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.298279 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.298257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.298541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.298526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.303092 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.303068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cwnv\" (UniqueName: \"kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv\") pod \"e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") " pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.357981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.357959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:39.633290 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.633215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:52:39.636075 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:52:39.636045    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6273ba09_6cb2_4829_a7cc_0e79349987d3.slice/crio-bc4a3aed9edc0192bef2188d654769c16771be87316039a885a7f3fc11f7062f WatchSource:0}: Error finding container bc4a3aed9edc0192bef2188d654769c16771be87316039a885a7f3fc11f7062f: Status 404 returned error can't find the container with id bc4a3aed9edc0192bef2188d654769c16771be87316039a885a7f3fc11f7062f
Sep 11 04:52:39.761740 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.761697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" event={"ID":"6273ba09-6cb2-4829-a7cc-0e79349987d3","Type":"ContainerStarted","Data":"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"}
Sep 11 04:52:39.761740 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.761742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" event={"ID":"6273ba09-6cb2-4829-a7cc-0e79349987d3","Type":"ContainerStarted","Data":"bc4a3aed9edc0192bef2188d654769c16771be87316039a885a7f3fc11f7062f"}
Sep 11 04:52:39.779080 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:39.779041    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" podStartSLOduration=0.779025935 podStartE2EDuration="779.025935ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:39.777955452 +0000 UTC m=+754.069429890" watchObservedRunningTime="2026-09-11 04:52:39.779025935 +0000 UTC m=+754.070500371"
Sep 11 04:52:49.359001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:49.358961    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:49.359001 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:49.359009    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:49.371412 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:49.371383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:49.854882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:49.854857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:52:53.790729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:53.790698    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:52:53.792024 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:53.790776    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:52:54.667068 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.667039    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:52:54.669273 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.669257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.671467 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.671443    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7bdb3c-openshift-default-dockercfg-k4jtv\""
Sep 11 04:52:54.686815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.682620    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:52:54.837331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.837697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.837596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqg9c\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.847698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.847671    2581 generic.go:356] "Generic (PLEG): container finished" podID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerID="2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb" exitCode=0
Sep 11 04:52:54.847821 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.847752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerDied","Data":"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"}
Sep 11 04:52:54.848182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.848164    2581 scope.go:117] "RemoveContainer" containerID="2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"
Sep 11 04:52:54.849188 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.849162    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:54.849283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.849231    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:54.849283 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.849264    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:54.938770 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.938747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.938879 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.938793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.938879 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.938819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.938879 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.938863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqg9c\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939089 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.938942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939089 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939089 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939870 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.939870 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.940013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.940013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.939944    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.941592 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.941565    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.942257 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.942239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.949124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.949104    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.949315 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.949275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqg9c\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c\") pod \"e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") " pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:54.980098 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:54.980073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:55.259781 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.259758    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:52:55.261049 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:52:55.261019    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f4bf76_0202_468c_aa53_1de6d690eb82.slice/crio-b4d3f29368b2988a3c4c1c01aa42d778d7c5a4d1b9d1a759773e802f4bb3936c WatchSource:0}: Error finding container b4d3f29368b2988a3c4c1c01aa42d778d7c5a4d1b9d1a759773e802f4bb3936c: Status 404 returned error can't find the container with id b4d3f29368b2988a3c4c1c01aa42d778d7c5a4d1b9d1a759773e802f4bb3936c
Sep 11 04:52:55.263226 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.263187    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:55.263346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.263271    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:55.263346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.263312    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:52:55.854569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.854534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerStarted","Data":"a7c1da84d46ee3c304496a89a21b2a467b852bd94504095ab25630288687d3ed"}
Sep 11 04:52:55.855024 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.854618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerStarted","Data":"b4d3f29368b2988a3c4c1c01aa42d778d7c5a4d1b9d1a759773e802f4bb3936c"}
Sep 11 04:52:55.856521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.856495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerStarted","Data":"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"}
Sep 11 04:52:55.874154 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.874097    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podStartSLOduration=1.8740793230000001 podStartE2EDuration="1.874079323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:55.872906531 +0000 UTC m=+770.164380969" watchObservedRunningTime="2026-09-11 04:52:55.874079323 +0000 UTC m=+770.165553760"
Sep 11 04:52:55.981020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:55.980992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:56.405214 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:56.405174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:52:56.405406 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:56.405387    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 04:52:56.405467 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:56.405432    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 04:52:56.985947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:56.985920    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:57.406118 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:57.406020    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 04:52:57.406118 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:57.406092    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 04:52:57.867258 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:57.867228    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:57.868205 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:57.868188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:52:58.405635 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:58.405602    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 04:52:58.405998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:58.405654    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 04:52:59.405961 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:59.405931    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 04:52:59.406310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:52:59.405987    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 04:53:00.405484 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:00.405450    2581 patch_prober.go:28] interesting pod/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 04:53:00.405794 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:00.405513    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 04:53:01.409527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:01.409497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:53:01.409921 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:01.409871    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:53:01.410670 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:01.410648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:53:02.123477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.123449    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"]
Sep 11 04:53:02.126769 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.126753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:02.128855 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.128817    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:53:02.129109 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.129094    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:53:02.134155 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.134132    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"]
Sep 11 04:53:02.200320 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.200289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d\" (UID: \"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:02.300762 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.300737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d\" (UID: \"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:02.301349 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.301328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d\" (UID: \"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:02.441323 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.441248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:02.710184 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.710161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"]
Sep 11 04:53:02.712251 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:53:02.712216    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77bb474d_2193_4ff2_9ac9_9f0c3dda6fbd.slice/crio-3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227 WatchSource:0}: Error finding container 3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227: Status 404 returned error can't find the container with id 3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227
Sep 11 04:53:02.893960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:02.893927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d" event={"ID":"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd","Type":"ContainerStarted","Data":"3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227"}
Sep 11 04:53:05.912068 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:05.911975    2581 generic.go:356] "Generic (PLEG): container finished" podID="77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" containerID="51ca7033b01efaf5948beb19975a4d02bc3234d59159f2c8f69c299eca4717c9" exitCode=0
Sep 11 04:53:05.912422 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:05.912058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d" event={"ID":"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd","Type":"ContainerDied","Data":"51ca7033b01efaf5948beb19975a4d02bc3234d59159f2c8f69c299eca4717c9"}
Sep 11 04:53:07.041088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.041063    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:07.144131 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.144103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca\") pod \"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd\" (UID: \"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd\") "
Sep 11 04:53:07.144496 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.144475    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" (UID: "77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:07.245562 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.245538    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:07.924008 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.923964    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d" event={"ID":"77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd","Type":"ContainerDied","Data":"3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227"}
Sep 11 04:53:07.924166 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.924025    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3af8b161f148d3ac27866ce54862d7c0bcccd3cfc21f97011db5b7dc19033227"
Sep 11 04:53:07.924166 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:07.923989    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"
Sep 11 04:53:17.988151 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.988122    2581 generic.go:356] "Generic (PLEG): container finished" podID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerID="a7c1da84d46ee3c304496a89a21b2a467b852bd94504095ab25630288687d3ed" exitCode=0
Sep 11 04:53:17.988527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.988180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerDied","Data":"a7c1da84d46ee3c304496a89a21b2a467b852bd94504095ab25630288687d3ed"}
Sep 11 04:53:17.988634 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.988574    2581 scope.go:117] "RemoveContainer" containerID="a7c1da84d46ee3c304496a89a21b2a467b852bd94504095ab25630288687d3ed"
Sep 11 04:53:17.989810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.989782    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:17.989881 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.989865    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:17.989923 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:17.989905    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:18.800818 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:18.800788    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"]
Sep 11 04:53:18.803528 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:18.803504    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5fa7070e-5dv5d"]
Sep 11 04:53:18.995562 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:18.995529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerStarted","Data":"162786e2b8ea0727611489cb5d672d76875fe030f393fd332644f109d408fa4a"}
Sep 11 04:53:19.833604 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.831540    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:53:19.840604 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.836892    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" containerName="revoke-keys"
Sep 11 04:53:19.840604 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.836922    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" containerName="revoke-keys"
Sep 11 04:53:19.840604 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.837014    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" containerName="revoke-keys"
Sep 11 04:53:19.851833 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.851803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:53:19.853044 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.852129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.857073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.857053    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1ecf5332-gw-openshift-default-dockercfg-bqjhk\""
Sep 11 04:53:19.964633 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.964837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.964837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.964837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.964837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.964837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.965053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8dr\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.965053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.965053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.964966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:19.980485 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.980461    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:53:19.980794 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.980773    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 11 04:53:19.980874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:19.980828    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 11 04:53:20.065763 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.065729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.065763 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.065762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.065811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.065991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066264 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8dr\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.066729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.066565    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.068523 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.068496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.068668 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.068651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.073780 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.073754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.073996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.073979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8dr\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr\") pod \"e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") " pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.168335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.168266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:20.192409 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.192382    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd" path="/var/lib/kubelet/pods/77bb474d-2193-4ff2-9ac9-9f0c3dda6fbd/volumes"
Sep 11 04:53:20.456404 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.456379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:53:20.457748 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:53:20.457722    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717d4f44_695b_4a45_bb31_0efd095812d0.slice/crio-cf38184b6e4459a7c318ccf93cfb67b4f8203cd45b17b3612e17e6563aa056c4 WatchSource:0}: Error finding container cf38184b6e4459a7c318ccf93cfb67b4f8203cd45b17b3612e17e6563aa056c4: Status 404 returned error can't find the container with id cf38184b6e4459a7c318ccf93cfb67b4f8203cd45b17b3612e17e6563aa056c4
Sep 11 04:53:20.460024 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.459991    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:20.460112 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.460058    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:20.460112 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.460087    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:20.981432 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.981397    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 11 04:53:20.981649 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:20.981453    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 11 04:53:21.008282 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.008252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerStarted","Data":"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"}
Sep 11 04:53:21.008419 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.008301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerStarted","Data":"cf38184b6e4459a7c318ccf93cfb67b4f8203cd45b17b3612e17e6563aa056c4"}
Sep 11 04:53:21.169022 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.168990    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:21.173711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.173690    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:21.192693 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.192654    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podStartSLOduration=2.192638936 podStartE2EDuration="2.192638936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:21.0306969 +0000 UTC m=+795.322171350" watchObservedRunningTime="2026-09-11 04:53:21.192638936 +0000 UTC m=+795.484113373"
Sep 11 04:53:21.981212 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.981172    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 11 04:53:21.981362 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:21.981232    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 11 04:53:22.014071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:22.014040    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:22.014935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:22.014918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:22.980820 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:22.980783    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 11 04:53:22.981298 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:22.980848    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 11 04:53:23.980870 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:23.980828    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 11 04:53:23.981244 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:23.980894    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 11 04:53:24.984421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:24.984394    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:53:24.984822 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:24.984696    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:53:24.985310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:24.985294    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:53:31.254817 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.254740    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:53:31.257997 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.257978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.260205 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.260182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-a0a72050-kserve-self-signed-certs\""
Sep 11 04:53:31.266985 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.266965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:53:31.361177 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.361177 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.361380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.361380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.361380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.361482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.361400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2xhp\" (UniqueName: \"kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462704 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462676    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462704 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462795    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.462890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.462811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2xhp\" (UniqueName: \"kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.463106 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.463087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.463174 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.463131    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.463231 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.463183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.465162 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.465143    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.465428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.465410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.470257 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.470233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2xhp\" (UniqueName: \"kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp\") pod \"e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") " pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.568514 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.568456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:31.848773 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:31.848751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:53:31.850678 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:53:31.850656    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f470e4e_bcc2_4f7d_a931_9b7dfd0608c9.slice/crio-4fe97cee56875c8e39d929e4d8376ae969633a6c856b1e00c94c93528126e1df WatchSource:0}: Error finding container 4fe97cee56875c8e39d929e4d8376ae969633a6c856b1e00c94c93528126e1df: Status 404 returned error can't find the container with id 4fe97cee56875c8e39d929e4d8376ae969633a6c856b1e00c94c93528126e1df
Sep 11 04:53:32.069737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:32.069703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" event={"ID":"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9","Type":"ContainerStarted","Data":"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"}
Sep 11 04:53:32.069902 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:32.069748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" event={"ID":"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9","Type":"ContainerStarted","Data":"4fe97cee56875c8e39d929e4d8376ae969633a6c856b1e00c94c93528126e1df"}
Sep 11 04:53:32.084914 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:32.084865    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" podStartSLOduration=1.08484604 podStartE2EDuration="1.08484604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:32.083877857 +0000 UTC m=+806.375352293" watchObservedRunningTime="2026-09-11 04:53:32.08484604 +0000 UTC m=+806.376320495"
Sep 11 04:53:41.569544 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:41.569517    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:41.569878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:41.569606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:41.582043 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:41.582025    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:42.136347 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:42.136312    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:53:45.143526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.143431    2581 generic.go:356] "Generic (PLEG): container finished" podID="717d4f44-695b-4a45-bb31-0efd095812d0" containerID="2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4" exitCode=0
Sep 11 04:53:45.143526 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.143498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerDied","Data":"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"}
Sep 11 04:53:45.143998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.143916    2581 scope.go:117] "RemoveContainer" containerID="2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"
Sep 11 04:53:45.145037 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.145014    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:45.145099 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.145075    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:45.145134 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:45.145108    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:53:46.152394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:46.152363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerStarted","Data":"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"}
Sep 11 04:53:46.168989 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:46.168965    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:46.169223 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:46.169195    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:46.169272 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:46.169247    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:47.168618 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.168565    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:47.168990 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.168639    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:47.363654 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.363620    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"]
Sep 11 04:53:47.366433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.366412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:47.368482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.368452    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:53:47.368626 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.368525    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:53:47.373329 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.373307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"]
Sep 11 04:53:47.516137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.516102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5\" (UID: \"a0d72c5e-9992-4008-ba31-96e3b8cc23f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:47.616913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.616871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5\" (UID: \"a0d72c5e-9992-4008-ba31-96e3b8cc23f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:47.617545 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.617525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5\" (UID: \"a0d72c5e-9992-4008-ba31-96e3b8cc23f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:47.678041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.677992    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:47.969552 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:47.969506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"]
Sep 11 04:53:47.971701 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:53:47.971660    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d72c5e_9992_4008_ba31_96e3b8cc23f1.slice/crio-fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1 WatchSource:0}: Error finding container fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1: Status 404 returned error can't find the container with id fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1
Sep 11 04:53:48.168792 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.168759    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:48.169192 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.168818    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:48.169939 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.169915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5" event={"ID":"a0d72c5e-9992-4008-ba31-96e3b8cc23f1","Type":"ContainerStarted","Data":"fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1"}
Sep 11 04:53:48.955635 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.955601    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:53:48.959245 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.959220    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:48.961505 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.961484    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0a8fef\"/\"kube-root-ca.crt\""
Sep 11 04:53:48.962137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.962113    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0a8fef\"/\"openshift-service-ca.crt\""
Sep 11 04:53:48.962243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.962122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0a8fef\"/\"default-dockercfg-9bq89\""
Sep 11 04:53:48.962243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.962207    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0a8fef\"/\"autores-model-0a8fef-kserve-self-signed-certs\""
Sep 11 04:53:48.968641 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:48.968619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:53:49.140156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.140156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjdtb\" (UniqueName: \"kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.140156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.140447 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.140447 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.140447 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.140337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.169329 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.169301    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:49.169759 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.169674    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:49.178291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.178264    2581 generic.go:356] "Generic (PLEG): container finished" podID="a0d72c5e-9992-4008-ba31-96e3b8cc23f1" containerID="80e53b4d4a787c4e830d60dc1761f794f6cb93b9f6683c463ce41195f85e8d83" exitCode=0
Sep 11 04:53:49.178462 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.178352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5" event={"ID":"a0d72c5e-9992-4008-ba31-96e3b8cc23f1","Type":"ContainerDied","Data":"80e53b4d4a787c4e830d60dc1761f794f6cb93b9f6683c463ce41195f85e8d83"}
Sep 11 04:53:49.241969 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.241933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.242236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.242211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjdtb\" (UniqueName: \"kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.242393 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.242377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.242538 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.242524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.242796 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.242776    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.242915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.242901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.243460 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.243437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.243861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.243840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.244625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.244555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.247641 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.247271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.247834 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.247810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.251331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.251311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjdtb\" (UniqueName: \"kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb\") pod \"autores-model-0a8fef-kserve-5f49854cd7-6xpfk\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") " pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.269363 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.269342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:49.542922 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:49.541161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:53:50.169305 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.169273    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:50.169477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.169323    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:50.185069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.185035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" event={"ID":"56a6a308-a73e-459d-947b-dec511b77f62","Type":"ContainerStarted","Data":"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"}
Sep 11 04:53:50.185233 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.185084    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" event={"ID":"56a6a308-a73e-459d-947b-dec511b77f62","Type":"ContainerStarted","Data":"9e789288c644bfe069ecb129309558d59c19eeaf7bdc7aea892a4a810d32ee59"}
Sep 11 04:53:50.201125 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.201068    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" podStartSLOduration=2.201047238 podStartE2EDuration="2.201047238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:50.199423878 +0000 UTC m=+824.490898316" watchObservedRunningTime="2026-09-11 04:53:50.201047238 +0000 UTC m=+824.492521677"
Sep 11 04:53:50.353252 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.353229    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:50.465157 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.465123    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca\") pod \"a0d72c5e-9992-4008-ba31-96e3b8cc23f1\" (UID: \"a0d72c5e-9992-4008-ba31-96e3b8cc23f1\") "
Sep 11 04:53:50.465443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.465419    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0d72c5e-9992-4008-ba31-96e3b8cc23f1" (UID: "a0d72c5e-9992-4008-ba31-96e3b8cc23f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:50.566405 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:50.566377    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d72c5e-9992-4008-ba31-96e3b8cc23f1-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:51.168860 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:51.168829    2581 patch_prober.go:28] interesting pod/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 04:53:51.169027 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:51.168885    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 04:53:51.192395 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:51.192369    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"
Sep 11 04:53:51.192815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:51.192416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5" event={"ID":"a0d72c5e-9992-4008-ba31-96e3b8cc23f1","Type":"ContainerDied","Data":"fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1"}
Sep 11 04:53:51.192815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:51.192455    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdcb236d6da9ff5c76be58c2fc75775486bc9933b76c68410a21ddad71b5fae1"
Sep 11 04:53:52.014981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:52.014955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:52.172411 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:52.172378    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:52.198662 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:52.198639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:54.196466 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:54.195530    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"]
Sep 11 04:53:54.199803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:54.199781    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ecf5332-5dmg5"]
Sep 11 04:53:54.326644 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:54.326609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:53:54.326884 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:54.326829    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy" containerID="cri-o://ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc" gracePeriod=30
Sep 11 04:53:56.193905 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.193872    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0d72c5e-9992-4008-ba31-96e3b8cc23f1" path="/var/lib/kubelet/pods/a0d72c5e-9992-4008-ba31-96e3b8cc23f1/volumes"
Sep 11 04:53:56.235382 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.235355    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:53:56.235780 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.235763    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0d72c5e-9992-4008-ba31-96e3b8cc23f1" containerName="revoke-keys"
Sep 11 04:53:56.235882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.235783    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0d72c5e-9992-4008-ba31-96e3b8cc23f1" containerName="revoke-keys"
Sep 11 04:53:56.235934 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.235890    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0d72c5e-9992-4008-ba31-96e3b8cc23f1" containerName="revoke-keys"
Sep 11 04:53:56.239877 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.239859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.243037 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.243011    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-831a1c\"/\"openshift-service-ca.crt\""
Sep 11 04:53:56.243129 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.243036    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-831a1c\"/\"kube-root-ca.crt\""
Sep 11 04:53:56.243129 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.243014    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-831a1c\"/\"test-model-831a1c-kserve-self-signed-certs\""
Sep 11 04:53:56.243129 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.243013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-831a1c\"/\"default-dockercfg-zrp9h\""
Sep 11 04:53:56.249090 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.249073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:53:56.306679 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.306831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.306831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbz8j\" (UniqueName: \"kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.306831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.306831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.307005 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.306851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.407734 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.407864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.407864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbz8j\" (UniqueName: \"kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.407864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.408034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.408034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.407920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.408137 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.408094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.408276 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.408257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.408356 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.408264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.410150 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.410119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.410431 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.410413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.415023 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.415005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbz8j\" (UniqueName: \"kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j\") pod \"test-model-831a1c-kserve-794854cbbc-52m66\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") " pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.550219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.550193    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:53:56.822444 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:56.822420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:53:56.823802 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:53:56.823776    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85b1e221_bb5d_44b5_a6e4_94d0b137a8ba.slice/crio-860ec4d44774533cafc7fac67da1599b1623deaf9fa9835e780e51190790b977 WatchSource:0}: Error finding container 860ec4d44774533cafc7fac67da1599b1623deaf9fa9835e780e51190790b977: Status 404 returned error can't find the container with id 860ec4d44774533cafc7fac67da1599b1623deaf9fa9835e780e51190790b977
Sep 11 04:53:57.224772 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:57.224736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" event={"ID":"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba","Type":"ContainerStarted","Data":"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"}
Sep 11 04:53:57.224772 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:57.224776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" event={"ID":"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba","Type":"ContainerStarted","Data":"860ec4d44774533cafc7fac67da1599b1623deaf9fa9835e780e51190790b977"}
Sep 11 04:53:57.243029 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:57.242984    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" podStartSLOduration=1.2429688859999999 podStartE2EDuration="1.242968886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:57.242162004 +0000 UTC m=+831.533636441" watchObservedRunningTime="2026-09-11 04:53:57.242968886 +0000 UTC m=+831.534443323"
Sep 11 04:53:59.270138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.270104    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:59.270511 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.270159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:59.282737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.282712    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:53:59.580417 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.580395    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:53:59.636604 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636745 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636745 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp8dr\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636808    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.636875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:53:59.637069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636876    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.637069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.637069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token\") pod \"717d4f44-695b-4a45-bb31-0efd095812d0\" (UID: \"717d4f44-695b-4a45-bb31-0efd095812d0\") "
Sep 11 04:53:59.637069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636966    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:53:59.637069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.636982    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:53:59.637340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.637236    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.637340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.637254    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.637340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.637251    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:53:59.637340 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.637267    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.637480 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.637331    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:59.639246 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.639197    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr" (OuterVolumeSpecName: "kube-api-access-bp8dr") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "kube-api-access-bp8dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:59.639352 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.639251    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:53:59.639522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.639503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:53:59.639601 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.639521    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "717d4f44-695b-4a45-bb31-0efd095812d0" (UID: "717d4f44-695b-4a45-bb31-0efd095812d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:59.734878 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.734852    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:53:59.738096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738047    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/717d4f44-695b-4a45-bb31-0efd095812d0-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.738096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738075    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.738096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738089    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp8dr\" (UniqueName: \"kubernetes.io/projected/717d4f44-695b-4a45-bb31-0efd095812d0-kube-api-access-bp8dr\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.738096 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738102    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.738377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738113    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/717d4f44-695b-4a45-bb31-0efd095812d0-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:53:59.738377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:53:59.738124    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/717d4f44-695b-4a45-bb31-0efd095812d0-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:00.247151 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.247123    2581 generic.go:356] "Generic (PLEG): container finished" podID="717d4f44-695b-4a45-bb31-0efd095812d0" containerID="ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc" exitCode=0
Sep 11 04:54:00.247331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.247165    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"
Sep 11 04:54:00.247331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.247205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerDied","Data":"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"}
Sep 11 04:54:00.247331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.247265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq" event={"ID":"717d4f44-695b-4a45-bb31-0efd095812d0","Type":"ContainerDied","Data":"cf38184b6e4459a7c318ccf93cfb67b4f8203cd45b17b3612e17e6563aa056c4"}
Sep 11 04:54:00.247331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.247294    2581 scope.go:117] "RemoveContainer" containerID="ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"
Sep 11 04:54:00.256928 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.256906    2581 scope.go:117] "RemoveContainer" containerID="2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"
Sep 11 04:54:00.260702 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.260647    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:54:00.263620 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.263600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:54:00.267018 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.266999    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1ecf5332-gw-openshift-default-87d59465b-98kpq"]
Sep 11 04:54:00.267870 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.267850    2581 scope.go:117] "RemoveContainer" containerID="ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"
Sep 11 04:54:00.268149 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:00.268123    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc\": container with ID starting with ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc not found: ID does not exist" containerID="ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"
Sep 11 04:54:00.268204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.268157    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc"} err="failed to get container status \"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc\": rpc error: code = NotFound desc = could not find container \"ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc\": container with ID starting with ba2e029291bbc8c0718ef3b8cd3250528c26ad6a5d43072a4decbd556349b5dc not found: ID does not exist"
Sep 11 04:54:00.268204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.268172    2581 scope.go:117] "RemoveContainer" containerID="2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"
Sep 11 04:54:00.268439 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:00.268422    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4\": container with ID starting with 2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4 not found: ID does not exist" containerID="2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"
Sep 11 04:54:00.268498 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:00.268448    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4"} err="failed to get container status \"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4\": rpc error: code = NotFound desc = could not find container \"2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4\": container with ID starting with 2a7d5d36a82a5d46b0c5b841d818c0c7720e1d85f7e778eb51701aa5f6817ec4 not found: ID does not exist"
Sep 11 04:54:02.192192 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.192160    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" path="/var/lib/kubelet/pods/717d4f44-695b-4a45-bb31-0efd095812d0/volumes"
Sep 11 04:54:02.429738 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.429709    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:54:02.547779 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.547745    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"]
Sep 11 04:54:02.548077 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548067    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.548124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548079    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.548124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548094    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.548124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548100    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.548223 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548164    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.548223 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.548171    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="717d4f44-695b-4a45-bb31-0efd095812d0" containerName="istio-proxy"
Sep 11 04:54:02.550883 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.550868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:02.552854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.552836    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:54:02.552955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.552845    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:54:02.567806 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.567549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"]
Sep 11 04:54:02.665644 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.665621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-0a8fef-l7djl\" (UID: \"9ea0d019-bc93-4212-8ea5-7732ebca79bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:02.766571 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.766538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-0a8fef-l7djl\" (UID: \"9ea0d019-bc93-4212-8ea5-7732ebca79bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:02.767139 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.767119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-0a8fef-l7djl\" (UID: \"9ea0d019-bc93-4212-8ea5-7732ebca79bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:02.865449 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:02.865387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:03.138688 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:03.138661    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"]
Sep 11 04:54:03.140132 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:54:03.140108    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea0d019_bc93_4212_8ea5_7732ebca79bb.slice/crio-3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f WatchSource:0}: Error finding container 3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f: Status 404 returned error can't find the container with id 3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f
Sep 11 04:54:03.265996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:03.265967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl" event={"ID":"9ea0d019-bc93-4212-8ea5-7732ebca79bb","Type":"ContainerStarted","Data":"3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f"}
Sep 11 04:54:04.272421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:04.272388    2581 generic.go:356] "Generic (PLEG): container finished" podID="9ea0d019-bc93-4212-8ea5-7732ebca79bb" containerID="f92e880b82fef43fbcda6ed7e9f2056c1428caf4a12179ce6c0f7d2e1275a1ba" exitCode=0
Sep 11 04:54:04.272831 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:04.272456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl" event={"ID":"9ea0d019-bc93-4212-8ea5-7732ebca79bb","Type":"ContainerDied","Data":"f92e880b82fef43fbcda6ed7e9f2056c1428caf4a12179ce6c0f7d2e1275a1ba"}
Sep 11 04:54:05.409306 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:05.409284    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:05.491946 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:05.491917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca\") pod \"9ea0d019-bc93-4212-8ea5-7732ebca79bb\" (UID: \"9ea0d019-bc93-4212-8ea5-7732ebca79bb\") "
Sep 11 04:54:05.492230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:05.492209    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ea0d019-bc93-4212-8ea5-7732ebca79bb" (UID: "9ea0d019-bc93-4212-8ea5-7732ebca79bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:05.592569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:05.592510    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0d019-bc93-4212-8ea5-7732ebca79bb-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:06.283744 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.283720    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"
Sep 11 04:54:06.283744 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.283728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl" event={"ID":"9ea0d019-bc93-4212-8ea5-7732ebca79bb","Type":"ContainerDied","Data":"3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f"}
Sep 11 04:54:06.283889 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.283768    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3297980fa1621feb3546c02124bd543e268ec8d2e02e04b008122f0730c4331f"
Sep 11 04:54:06.550868 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.550841    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:54:06.551234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.550882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:54:06.563172 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:06.563152    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:54:07.299689 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:07.299656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:54:13.162386 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:13.162354    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"]
Sep 11 04:54:13.168656 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:13.168635    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-0a8fef-l7djl"]
Sep 11 04:54:14.192830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:14.192795    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ea0d019-bc93-4212-8ea5-7732ebca79bb" path="/var/lib/kubelet/pods/9ea0d019-bc93-4212-8ea5-7732ebca79bb/volumes"
Sep 11 04:54:14.754830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:14.754798    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" podUID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" containerName="main" containerID="cri-o://a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb" gracePeriod=45
Sep 11 04:54:15.005179 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.005125    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:54:15.071914 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.071887    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2xhp\" (UniqueName: \"kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072079 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.071935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072079 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072003    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072079 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072059    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072245 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072111    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072245 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home\") pod \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\" (UID: \"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9\") "
Sep 11 04:54:15.072347 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:15.072515 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072489    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home" (OuterVolumeSpecName: "home") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:15.072679 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072551    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.072742 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.072681    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:15.074532 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.074509    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:54:15.074659 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.074557    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm" (OuterVolumeSpecName: "dshm") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:15.074716 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.074697    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp" (OuterVolumeSpecName: "kube-api-access-v2xhp") pod "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" (UID: "9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9"). InnerVolumeSpecName "kube-api-access-v2xhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:15.173720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.173676    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.173720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.173715    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.173720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.173728    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.173947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.173740    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.173947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.173751    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2xhp\" (UniqueName: \"kubernetes.io/projected/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9-kube-api-access-v2xhp\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:15.331082 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.330998    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" containerID="a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb" exitCode=0
Sep 11 04:54:15.331082 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.331041    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"
Sep 11 04:54:15.331575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.331087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" event={"ID":"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9","Type":"ContainerDied","Data":"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"}
Sep 11 04:54:15.331575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.331130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq" event={"ID":"9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9","Type":"ContainerDied","Data":"4fe97cee56875c8e39d929e4d8376ae969633a6c856b1e00c94c93528126e1df"}
Sep 11 04:54:15.331575 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.331151    2581 scope.go:117] "RemoveContainer" containerID="a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"
Sep 11 04:54:15.340360 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.340345    2581 scope.go:117] "RemoveContainer" containerID="a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"
Sep 11 04:54:15.340664 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:15.340645    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb\": container with ID starting with a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb not found: ID does not exist" containerID="a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"
Sep 11 04:54:15.340738 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.340671    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb"} err="failed to get container status \"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb\": rpc error: code = NotFound desc = could not find container \"a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb\": container with ID starting with a87d4c7bc8e9470f7113a3638dc6693bd44c370c218e5d2ebe944a3d73850fcb not found: ID does not exist"
Sep 11 04:54:15.351028 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.351002    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:54:15.353107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:15.353090    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-a0a72050-kserve-6f58fdc469-km5rq"]
Sep 11 04:54:16.158652 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:16.158620    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:54:16.193122 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:16.193089    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" path="/var/lib/kubelet/pods/9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9/volumes"
Sep 11 04:54:17.449434 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.449389    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" podUID="56a6a308-a73e-459d-947b-dec511b77f62" containerName="main" containerID="cri-o://5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d" gracePeriod=45
Sep 11 04:54:17.700828 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.700770    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:54:17.796268 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796429 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796310    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796429 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796333    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796429 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796355    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796429 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796429 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjdtb\" (UniqueName: \"kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb\") pod \"56a6a308-a73e-459d-947b-dec511b77f62\" (UID: \"56a6a308-a73e-459d-947b-dec511b77f62\") "
Sep 11 04:54:17.796722 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:17.796722 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796660    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home" (OuterVolumeSpecName: "home") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:17.796722 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796669    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache" (OuterVolumeSpecName: "model-cache") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:17.796882 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.796746    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:17.798751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.798726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm" (OuterVolumeSpecName: "dshm") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:17.798751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.798742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb" (OuterVolumeSpecName: "kube-api-access-vjdtb") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "kube-api-access-vjdtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:17.798883 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.798776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56a6a308-a73e-459d-947b-dec511b77f62" (UID: "56a6a308-a73e-459d-947b-dec511b77f62"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:54:17.897531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.897505    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:17.897531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.897527    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a6a308-a73e-459d-947b-dec511b77f62-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:17.897698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.897536    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:17.897698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.897543    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a6a308-a73e-459d-947b-dec511b77f62-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:17.897698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:17.897551    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjdtb\" (UniqueName: \"kubernetes.io/projected/56a6a308-a73e-459d-947b-dec511b77f62-kube-api-access-vjdtb\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:18.352666 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.352632    2581 generic.go:356] "Generic (PLEG): container finished" podID="56a6a308-a73e-459d-947b-dec511b77f62" containerID="5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d" exitCode=0
Sep 11 04:54:18.352793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.352676    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"
Sep 11 04:54:18.352793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.352719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" event={"ID":"56a6a308-a73e-459d-947b-dec511b77f62","Type":"ContainerDied","Data":"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"}
Sep 11 04:54:18.352793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.352762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk" event={"ID":"56a6a308-a73e-459d-947b-dec511b77f62","Type":"ContainerDied","Data":"9e789288c644bfe069ecb129309558d59c19eeaf7bdc7aea892a4a810d32ee59"}
Sep 11 04:54:18.352793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.352780    2581 scope.go:117] "RemoveContainer" containerID="5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"
Sep 11 04:54:18.361988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.361969    2581 scope.go:117] "RemoveContainer" containerID="5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"
Sep 11 04:54:18.362254 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:18.362237    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d\": container with ID starting with 5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d not found: ID does not exist" containerID="5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"
Sep 11 04:54:18.362307 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.362263    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d"} err="failed to get container status \"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d\": rpc error: code = NotFound desc = could not find container \"5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d\": container with ID starting with 5f819cb691cc35cf57bf36db69a049b8c6964bb53f58528c7849eaef9588457d not found: ID does not exist"
Sep 11 04:54:18.368394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.368374    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:54:18.373446 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:18.373416    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-0a8fef/autores-model-0a8fef-kserve-5f49854cd7-6xpfk"]
Sep 11 04:54:19.622616 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:19.622561    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:54:19.623094 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:19.622832    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy" containerID="cri-o://950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c" gracePeriod=30
Sep 11 04:54:20.192777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:20.192744    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a6a308-a73e-459d-947b-dec511b77f62" path="/var/lib/kubelet/pods/56a6a308-a73e-459d-947b-dec511b77f62/volumes"
Sep 11 04:54:22.825182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825153    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"]
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825470    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" containerName="main"
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825482    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" containerName="main"
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825492    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea0d019-bc93-4212-8ea5-7732ebca79bb" containerName="revoke-keys"
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825498    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea0d019-bc93-4212-8ea5-7732ebca79bb" containerName="revoke-keys"
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825512    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a6a308-a73e-459d-947b-dec511b77f62" containerName="main"
Sep 11 04:54:22.825547 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825517    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a6a308-a73e-459d-947b-dec511b77f62" containerName="main"
Sep 11 04:54:22.825756 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825603    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f470e4e-bcc2-4f7d-a931-9b7dfd0608c9" containerName="main"
Sep 11 04:54:22.825756 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825614    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a6a308-a73e-459d-947b-dec511b77f62" containerName="main"
Sep 11 04:54:22.825756 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.825621    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea0d019-bc93-4212-8ea5-7732ebca79bb" containerName="revoke-keys"
Sep 11 04:54:22.829848 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.829832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:22.832073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.832050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:54:22.832073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.832071    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:54:22.836045 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.836023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"]
Sep 11 04:54:22.942025 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:22.941997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d2554f2-lb978\" (UID: \"4a8ea458-d2a4-4685-877a-da19abbde9ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:23.043365 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:23.043339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d2554f2-lb978\" (UID: \"4a8ea458-d2a4-4685-877a-da19abbde9ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:23.043952 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:23.043934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d2554f2-lb978\" (UID: \"4a8ea458-d2a4-4685-877a-da19abbde9ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:23.141293 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:23.141237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:23.409540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:23.409518    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"]
Sep 11 04:54:23.411255 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:54:23.411229    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8ea458_d2a4_4685_877a_da19abbde9ec.slice/crio-478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83 WatchSource:0}: Error finding container 478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83: Status 404 returned error can't find the container with id 478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83
Sep 11 04:54:24.386813 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.386743    2581 generic.go:356] "Generic (PLEG): container finished" podID="4a8ea458-d2a4-4685-877a-da19abbde9ec" containerID="71fac0a9ed191db46ea7e041a3d61f3ebb623166aba344cd22be5d25a13f20b5" exitCode=0
Sep 11 04:54:24.387107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.386807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978" event={"ID":"4a8ea458-d2a4-4685-877a-da19abbde9ec","Type":"ContainerDied","Data":"71fac0a9ed191db46ea7e041a3d61f3ebb623166aba344cd22be5d25a13f20b5"}
Sep 11 04:54:24.387107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.386849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978" event={"ID":"4a8ea458-d2a4-4685-877a-da19abbde9ec","Type":"ContainerStarted","Data":"478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83"}
Sep 11 04:54:24.862965 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.862944    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:54:24.957711 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957685    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957782    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957817    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957847    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.957900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.958228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvtcb\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.958228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.957996    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token\") pod \"99e10ac2-efe3-4d00-ac05-87521e28de2b\" (UID: \"99e10ac2-efe3-4d00-ac05-87521e28de2b\") "
Sep 11 04:54:24.958228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:24.958228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958198    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:24.958433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:24.958433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958387    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:24.958433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958411    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:24.958772 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958452    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:24.958772 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958428    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:24.958772 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.958661    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:24.960247 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.960223    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:24.960247 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.960229    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:54:24.960766 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.960744    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb" (OuterVolumeSpecName: "kube-api-access-tvtcb") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "kube-api-access-tvtcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:24.960766 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:24.960749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "99e10ac2-efe3-4d00-ac05-87521e28de2b" (UID: "99e10ac2-efe3-4d00-ac05-87521e28de2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:25.059674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059650    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.059674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059670    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e10ac2-efe3-4d00-ac05-87521e28de2b-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.059803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059679    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.059803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059689    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.059803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059698    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvtcb\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-kube-api-access-tvtcb\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.059803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.059707    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e10ac2-efe3-4d00-ac05-87521e28de2b-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:25.394006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.393914    2581 generic.go:356] "Generic (PLEG): container finished" podID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerID="950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c" exitCode=0
Sep 11 04:54:25.394006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.393971    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"
Sep 11 04:54:25.394466 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.394003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerDied","Data":"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"}
Sep 11 04:54:25.394466 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.394059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw" event={"ID":"99e10ac2-efe3-4d00-ac05-87521e28de2b","Type":"ContainerDied","Data":"9f38572d0d2c5e1ea41fe0a7bbb2a73b0a903887a07d5e3453945a7c9baed4a3"}
Sep 11 04:54:25.394466 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.394080    2581 scope.go:117] "RemoveContainer" containerID="950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"
Sep 11 04:54:25.404471 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.404456    2581 scope.go:117] "RemoveContainer" containerID="2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"
Sep 11 04:54:25.413904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.413870    2581 scope.go:117] "RemoveContainer" containerID="950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"
Sep 11 04:54:25.414251 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:25.414194    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c\": container with ID starting with 950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c not found: ID does not exist" containerID="950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"
Sep 11 04:54:25.414251 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.414224    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c"} err="failed to get container status \"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c\": rpc error: code = NotFound desc = could not find container \"950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c\": container with ID starting with 950dea623fad0ee30e76dcf92817066937cf40023b3d935e33ce18c47f506b3c not found: ID does not exist"
Sep 11 04:54:25.414251 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.414241    2581 scope.go:117] "RemoveContainer" containerID="2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"
Sep 11 04:54:25.414530 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:25.414498    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb\": container with ID starting with 2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb not found: ID does not exist" containerID="2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"
Sep 11 04:54:25.414667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.414536    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb"} err="failed to get container status \"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb\": rpc error: code = NotFound desc = could not find container \"2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb\": container with ID starting with 2a96fc296513375a3357c0626a085a9fe689dc7b7f026b5ee470a11565c73edb not found: ID does not exist"
Sep 11 04:54:25.418056 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.418027    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:54:25.420311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.420287    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-0a8fef-openshift-default-5d48c4d994-2c2bw"]
Sep 11 04:54:25.508653 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.508629    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:25.665182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.665120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca\") pod \"4a8ea458-d2a4-4685-877a-da19abbde9ec\" (UID: \"4a8ea458-d2a4-4685-877a-da19abbde9ec\") "
Sep 11 04:54:25.665438 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.665418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a8ea458-d2a4-4685-877a-da19abbde9ec" (UID: "4a8ea458-d2a4-4685-877a-da19abbde9ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:25.766228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:25.766198    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a8ea458-d2a4-4685-877a-da19abbde9ec-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:26.197462 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:26.197429    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" path="/var/lib/kubelet/pods/99e10ac2-efe3-4d00-ac05-87521e28de2b/volumes"
Sep 11 04:54:26.401234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:26.401198    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"
Sep 11 04:54:26.401234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:26.401226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978" event={"ID":"4a8ea458-d2a4-4685-877a-da19abbde9ec","Type":"ContainerDied","Data":"478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83"}
Sep 11 04:54:26.401695 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:26.401266    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="478ebbaf21c82edff5cb0355e6bd793caa74cac55b0355e8542d336b7cb0af83"
Sep 11 04:54:29.461380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:29.461347    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"]
Sep 11 04:54:29.463479 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:29.463459    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d2554f2-lb978"]
Sep 11 04:54:30.192650 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:30.192623    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a8ea458-d2a4-4685-877a-da19abbde9ec" path="/var/lib/kubelet/pods/4a8ea458-d2a4-4685-877a-da19abbde9ec/volumes"
Sep 11 04:54:31.177813 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.177774    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" podUID="6273ba09-6cb2-4829-a7cc-0e79349987d3" containerName="main" containerID="cri-o://bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce" gracePeriod=45
Sep 11 04:54:31.246808 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.246778    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:54:31.247087 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247073    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:31.247087 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247086    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247095    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247100    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247106    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a8ea458-d2a4-4685-877a-da19abbde9ec" containerName="revoke-keys"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247127    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a8ea458-d2a4-4685-877a-da19abbde9ec" containerName="revoke-keys"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247194    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:31.247228 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.247204    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a8ea458-d2a4-4685-877a-da19abbde9ec" containerName="revoke-keys"
Sep 11 04:54:31.250144 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.250126    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.252265 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.252249    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-fbfc153d-openshift-default-dockercfg-l2rpm\""
Sep 11 04:54:31.264951 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.264916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:54:31.410437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwl7\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.410837 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.410760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.428170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.428108    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:54:31.430570 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.430550    2581 generic.go:356] "Generic (PLEG): container finished" podID="6273ba09-6cb2-4829-a7cc-0e79349987d3" containerID="bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce" exitCode=0
Sep 11 04:54:31.430685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.430600    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"
Sep 11 04:54:31.430685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.430609    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" event={"ID":"6273ba09-6cb2-4829-a7cc-0e79349987d3","Type":"ContainerDied","Data":"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"}
Sep 11 04:54:31.430685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.430658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h" event={"ID":"6273ba09-6cb2-4829-a7cc-0e79349987d3","Type":"ContainerDied","Data":"bc4a3aed9edc0192bef2188d654769c16771be87316039a885a7f3fc11f7062f"}
Sep 11 04:54:31.430685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.430672    2581 scope.go:117] "RemoveContainer" containerID="bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"
Sep 11 04:54:31.439396 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.439379    2581 scope.go:117] "RemoveContainer" containerID="bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"
Sep 11 04:54:31.439667 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:54:31.439649    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce\": container with ID starting with bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce not found: ID does not exist" containerID="bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"
Sep 11 04:54:31.439739 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.439678    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce"} err="failed to get container status \"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce\": rpc error: code = NotFound desc = could not find container \"bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce\": container with ID starting with bcc068ca384740650d0ab03ca8c6a742f1cf62c8d785c51b47d8b9cd9a7558ce not found: ID does not exist"
Sep 11 04:54:31.511305 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511361    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cwnv\" (UniqueName: \"kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511504 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511401    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511504 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511440    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home\") pod \"6273ba09-6cb2-4829-a7cc-0e79349987d3\" (UID: \"6273ba09-6cb2-4829-a7cc-0e79349987d3\") "
Sep 11 04:54:31.511629 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.511689 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.511746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.511746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:31.511746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:31.511893 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.511893 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511843    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home" (OuterVolumeSpecName: "home") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:31.511893 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512042 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512042 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512042 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.511989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwl7\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512079    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512449    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512707    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512731    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.512774 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.512746    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.514111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.514088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm" (OuterVolumeSpecName: "dshm") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:54:31.514203 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.514159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.514295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.514268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:54:31.514522 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.514480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.514808 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.514787    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv" (OuterVolumeSpecName: "kube-api-access-2cwnv") pod "6273ba09-6cb2-4829-a7cc-0e79349987d3" (UID: "6273ba09-6cb2-4829-a7cc-0e79349987d3"). InnerVolumeSpecName "kube-api-access-2cwnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:31.520443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.520424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwl7\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.520520 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.520503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token\") pod \"e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") " pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.561320 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.561298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:31.614412 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.614380    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6273ba09-6cb2-4829-a7cc-0e79349987d3-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.614528 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.614421    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cwnv\" (UniqueName: \"kubernetes.io/projected/6273ba09-6cb2-4829-a7cc-0e79349987d3-kube-api-access-2cwnv\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.614528 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.614438    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6273ba09-6cb2-4829-a7cc-0e79349987d3-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:54:31.752201 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.752172    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:54:31.754284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.754263    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-a0a72050-kserve-547dcf7775-5p29h"]
Sep 11 04:54:31.832867 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.832842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:54:31.834412 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:54:31.834392    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefae565f_bd7c_4513_8410_b886dc6ad466.slice/crio-4561cff4bc0fdf2e2a4f367f070fca145728e3818eaf16f0bda019a7c635ae53 WatchSource:0}: Error finding container 4561cff4bc0fdf2e2a4f367f070fca145728e3818eaf16f0bda019a7c635ae53: Status 404 returned error can't find the container with id 4561cff4bc0fdf2e2a4f367f070fca145728e3818eaf16f0bda019a7c635ae53
Sep 11 04:54:31.836592 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.836553    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:31.836653 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.836640    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:31.836700 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:31.836674    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:32.192234 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.192150    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6273ba09-6cb2-4829-a7cc-0e79349987d3" path="/var/lib/kubelet/pods/6273ba09-6cb2-4829-a7cc-0e79349987d3/volumes"
Sep 11 04:54:32.437189 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.437151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerStarted","Data":"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"}
Sep 11 04:54:32.437358 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.437223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerStarted","Data":"4561cff4bc0fdf2e2a4f367f070fca145728e3818eaf16f0bda019a7c635ae53"}
Sep 11 04:54:32.457128 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.457085    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podStartSLOduration=1.4570704509999999 podStartE2EDuration="1.457070451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:32.454539957 +0000 UTC m=+866.746014407" watchObservedRunningTime="2026-09-11 04:54:32.457070451 +0000 UTC m=+866.748544888"
Sep 11 04:54:32.561793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.561770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:32.572261 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:32.572240    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:33.444755 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:33.444730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:33.446168 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:33.446147    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:49.486716 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:49.486686    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:54:49.487219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:49.486733    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:54:50.539574 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.539536    2581 generic.go:356] "Generic (PLEG): container finished" podID="efae565f-bd7c-4513-8410-b886dc6ad466" containerID="916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2" exitCode=0
Sep 11 04:54:50.540099 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.539612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerDied","Data":"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"}
Sep 11 04:54:50.540099 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.540068    2581 scope.go:117] "RemoveContainer" containerID="916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"
Sep 11 04:54:50.541448 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.541407    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:50.541606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.541515    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:50.541606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:50.541571    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:54:51.550753 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:51.550708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerStarted","Data":"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"}
Sep 11 04:54:51.561962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:51.561941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:51.562128 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:51.562110    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 04:54:51.562172 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:51.562152    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 04:54:52.562400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:52.562368    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 04:54:52.562813 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:52.562428    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 04:54:53.562006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:53.561969    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 04:54:53.562171 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:53.562039    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 04:54:54.561761 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:54.561728    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 04:54:54.562127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:54.561788    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 04:54:55.562453 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:55.562418    2581 patch_prober.go:28] interesting pod/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 04:54:55.562861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:55.562475    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 04:54:56.566793 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:56.566754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:56.567253 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:56.567158    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:56.567974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:56.567957    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:54:58.583555 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.583470    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"]
Sep 11 04:54:58.583955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.583877    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6273ba09-6cb2-4829-a7cc-0e79349987d3" containerName="main"
Sep 11 04:54:58.583955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.583891    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6273ba09-6cb2-4829-a7cc-0e79349987d3" containerName="main"
Sep 11 04:54:58.584045 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.583986    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e10ac2-efe3-4d00-ac05-87521e28de2b" containerName="istio-proxy"
Sep 11 04:54:58.584045 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.583993    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6273ba09-6cb2-4829-a7cc-0e79349987d3" containerName="main"
Sep 11 04:54:58.586182 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.586158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:54:58.588149 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.588129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:54:58.588256 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.588170    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:54:58.593353 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.593330    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"]
Sep 11 04:54:58.661630 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.661603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw\" (UID: \"183cd9ad-1e08-44f7-af5f-a040053a3ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:54:58.762971 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.762942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw\" (UID: \"183cd9ad-1e08-44f7-af5f-a040053a3ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:54:58.763566 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.763541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw\" (UID: \"183cd9ad-1e08-44f7-af5f-a040053a3ddb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:54:58.900287 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:58.900211    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:54:59.170625 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.170599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"]
Sep 11 04:54:59.172261 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:54:59.172231    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183cd9ad_1e08_44f7_af5f_a040053a3ddb.slice/crio-a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316 WatchSource:0}: Error finding container a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316: Status 404 returned error can't find the container with id a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316
Sep 11 04:54:59.596449 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.596417    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw" event={"ID":"183cd9ad-1e08-44f7-af5f-a040053a3ddb","Type":"ContainerStarted","Data":"cbf784db706453f8d25900b2215e8c8fa178a8e71bbe3342a305224fcb2dc8a0"}
Sep 11 04:54:59.596890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.596477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw" event={"ID":"183cd9ad-1e08-44f7-af5f-a040053a3ddb","Type":"ContainerStarted","Data":"a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316"}
Sep 11 04:54:59.612275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.612230    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw" podStartSLOduration=1.612214163 podStartE2EDuration="1.612214163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:59.609899561 +0000 UTC m=+893.901373999" watchObservedRunningTime="2026-09-11 04:54:59.612214163 +0000 UTC m=+893.903688599"
Sep 11 04:54:59.879794 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.879729    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:54:59.881995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.881981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.886944 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.886924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7bdb3c\"/\"default-dockercfg-tqz6r\""
Sep 11 04:54:59.886944 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.886937    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7bdb3c\"/\"kube-root-ca.crt\""
Sep 11 04:54:59.887134 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.886927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7bdb3c\"/\"test-model-7bdb3c-kserve-self-signed-certs\""
Sep 11 04:54:59.887435 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.887420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7bdb3c\"/\"openshift-service-ca.crt\""
Sep 11 04:54:59.911030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.911008    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:54:59.973669 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.973823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.973823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.973823 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.973990 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh6p4\" (UniqueName: \"kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:54:59.973990 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:54:59.973928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075271 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh6p4\" (UniqueName: \"kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075813 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.075876 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.075830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.077739 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.077723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.077994 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.077979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.082370 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.082346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh6p4\" (UniqueName: \"kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4\") pod \"test-model-7bdb3c-kserve-7dbfbdbd67-gm8db\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") " pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.190560 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.190497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:00.476276 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.476252    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:55:00.477191 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:55:00.477159    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc63042eb_3814_4711_8f11_e531d1506297.slice/crio-fde40cd5ae3886e138415e4395740d500cb02167a5d734430b9b2ff1a0f01f5a WatchSource:0}: Error finding container fde40cd5ae3886e138415e4395740d500cb02167a5d734430b9b2ff1a0f01f5a: Status 404 returned error can't find the container with id fde40cd5ae3886e138415e4395740d500cb02167a5d734430b9b2ff1a0f01f5a
Sep 11 04:55:00.603649 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.603619    2581 generic.go:356] "Generic (PLEG): container finished" podID="183cd9ad-1e08-44f7-af5f-a040053a3ddb" containerID="cbf784db706453f8d25900b2215e8c8fa178a8e71bbe3342a305224fcb2dc8a0" exitCode=0
Sep 11 04:55:00.604052 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.603701    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw" event={"ID":"183cd9ad-1e08-44f7-af5f-a040053a3ddb","Type":"ContainerDied","Data":"cbf784db706453f8d25900b2215e8c8fa178a8e71bbe3342a305224fcb2dc8a0"}
Sep 11 04:55:00.605601 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.605548    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" event={"ID":"c63042eb-3814-4711-8f11-e531d1506297","Type":"ContainerStarted","Data":"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"}
Sep 11 04:55:00.605601 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.605598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" event={"ID":"c63042eb-3814-4711-8f11-e531d1506297","Type":"ContainerStarted","Data":"fde40cd5ae3886e138415e4395740d500cb02167a5d734430b9b2ff1a0f01f5a"}
Sep 11 04:55:00.645681 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:00.645547    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" podStartSLOduration=1.6455296750000001 podStartE2EDuration="1.645529675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:55:00.635812296 +0000 UTC m=+894.927286758" watchObservedRunningTime="2026-09-11 04:55:00.645529675 +0000 UTC m=+894.937004111"
Sep 11 04:55:01.733400 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:01.733376    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:55:01.791541 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:01.791511    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca\") pod \"183cd9ad-1e08-44f7-af5f-a040053a3ddb\" (UID: \"183cd9ad-1e08-44f7-af5f-a040053a3ddb\") "
Sep 11 04:55:01.791840 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:01.791818    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "183cd9ad-1e08-44f7-af5f-a040053a3ddb" (UID: "183cd9ad-1e08-44f7-af5f-a040053a3ddb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:01.892395 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:01.892336    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/183cd9ad-1e08-44f7-af5f-a040053a3ddb-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:02.618443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:02.618359    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"
Sep 11 04:55:02.618443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:02.618381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw" event={"ID":"183cd9ad-1e08-44f7-af5f-a040053a3ddb","Type":"ContainerDied","Data":"a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316"}
Sep 11 04:55:02.618443 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:02.618423    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a696c0f9bc7fc1f9856fbc1caa038498f6030911eb8c8de7ae9cc0fb3c026316"
Sep 11 04:55:05.664281 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:05.664251    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"]
Sep 11 04:55:05.668222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:05.668201    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fbfc153d-q8hnw"]
Sep 11 04:55:05.796767 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:05.796739    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:55:05.796962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:05.796939    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy" containerID="cri-o://1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3" gracePeriod=30
Sep 11 04:55:06.169313 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:06.169285    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:55:06.172715 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:06.172693    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 04:55:06.192825 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:06.192799    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="183cd9ad-1e08-44f7-af5f-a040053a3ddb" path="/var/lib/kubelet/pods/183cd9ad-1e08-44f7-af5f-a040053a3ddb/volumes"
Sep 11 04:55:10.197159 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:10.197126    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:10.197565 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:10.197171    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:10.204945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:10.204924    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:10.673333 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:10.673258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:55:11.046243 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.046217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:55:11.166116 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166084    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxwl7\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166200    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166265    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy\") pod \"efae565f-bd7c-4513-8410-b886dc6ad466\" (UID: \"efae565f-bd7c-4513-8410-b886dc6ad466\") "
Sep 11 04:55:11.166706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data" (OuterVolumeSpecName: "istio-data") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:11.166706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166640    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:11.166706 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166645    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.166904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:11.166904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166752    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:11.166904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.166756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:11.168627 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.168559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7" (OuterVolumeSpecName: "kube-api-access-bxwl7") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "kube-api-access-bxwl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:11.168737 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.168651    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:11.168918 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.168899    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token" (OuterVolumeSpecName: "istio-token") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:11.168918 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.168907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efae565f-bd7c-4513-8410-b886dc6ad466" (UID: "efae565f-bd7c-4513-8410-b886dc6ad466"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267658    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efae565f-bd7c-4513-8410-b886dc6ad466-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267679    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267688    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267697    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267705    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efae565f-bd7c-4513-8410-b886dc6ad466-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.267714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267712    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.268156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267720    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efae565f-bd7c-4513-8410-b886dc6ad466-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.268156 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.267728    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxwl7\" (UniqueName: \"kubernetes.io/projected/efae565f-bd7c-4513-8410-b886dc6ad466-kube-api-access-bxwl7\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:11.669091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.669001    2581 generic.go:356] "Generic (PLEG): container finished" podID="efae565f-bd7c-4513-8410-b886dc6ad466" containerID="1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3" exitCode=0
Sep 11 04:55:11.669091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.669050    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"
Sep 11 04:55:11.669302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.669087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerDied","Data":"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"}
Sep 11 04:55:11.669302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.669134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm" event={"ID":"efae565f-bd7c-4513-8410-b886dc6ad466","Type":"ContainerDied","Data":"4561cff4bc0fdf2e2a4f367f070fca145728e3818eaf16f0bda019a7c635ae53"}
Sep 11 04:55:11.669302 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.669156    2581 scope.go:117] "RemoveContainer" containerID="1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"
Sep 11 04:55:11.680758 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.680734    2581 scope.go:117] "RemoveContainer" containerID="916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"
Sep 11 04:55:11.690107 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.690085    2581 scope.go:117] "RemoveContainer" containerID="1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"
Sep 11 04:55:11.690423 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:55:11.690398    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3\": container with ID starting with 1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3 not found: ID does not exist" containerID="1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"
Sep 11 04:55:11.690495 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.690423    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3"} err="failed to get container status \"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3\": rpc error: code = NotFound desc = could not find container \"1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3\": container with ID starting with 1e59168b37497da58d4cce6a748beda688da4b85254b09cdeaedd527fd5e49f3 not found: ID does not exist"
Sep 11 04:55:11.690495 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.690439    2581 scope.go:117] "RemoveContainer" containerID="916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"
Sep 11 04:55:11.690727 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:55:11.690709    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2\": container with ID starting with 916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2 not found: ID does not exist" containerID="916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"
Sep 11 04:55:11.690790 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.690736    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2"} err="failed to get container status \"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2\": rpc error: code = NotFound desc = could not find container \"916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2\": container with ID starting with 916612c01bc4d3b928969c2bc13e55ce27bf3a65a934518caa826582584c1da2 not found: ID does not exist"
Sep 11 04:55:11.691943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.691926    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:55:11.697104 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:11.697086    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-fbfc153d-openshift-default-76bc5854d8-whsrm"]
Sep 11 04:55:12.191974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:12.191944    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" path="/var/lib/kubelet/pods/efae565f-bd7c-4513-8410-b886dc6ad466/volumes"
Sep 11 04:55:17.755334 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755304    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755622    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="183cd9ad-1e08-44f7-af5f-a040053a3ddb" containerName="revoke-keys"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755633    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="183cd9ad-1e08-44f7-af5f-a040053a3ddb" containerName="revoke-keys"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755653    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755658    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755669    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755675    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755727    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:17.755746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.755735    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="183cd9ad-1e08-44f7-af5f-a040053a3ddb" containerName="revoke-keys"
Sep 11 04:55:17.760070 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.760050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.762452 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.762430    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-eefe49\"/\"openshift-service-ca.crt\""
Sep 11 04:55:17.763083 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.763064    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-eefe49\"/\"explicit-model-eefe49-kserve-self-signed-certs\""
Sep 11 04:55:17.763174 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.763073    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-eefe49\"/\"kube-root-ca.crt\""
Sep 11 04:55:17.763174 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.763103    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-eefe49\"/\"default-dockercfg-fxr29\""
Sep 11 04:55:17.770451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.770431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:17.824428 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.824553 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ccm9\" (UniqueName: \"kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.824553 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.824553 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.824749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.824749 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.824683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925392 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925502 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ccm9\" (UniqueName: \"kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.925954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.926041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.926041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.925986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.928091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.928067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.928620 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.928601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:17.933143 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:17.933123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ccm9\" (UniqueName: \"kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9\") pod \"explicit-model-eefe49-kserve-6848d9cc77-jn5gb\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") " pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:18.071074 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:18.071014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:18.343313 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:18.343286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:18.345046 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:55:18.345005    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00614f13_7a64_4725_a20d_7bb35825243b.slice/crio-3000b1d9a4b3cb34a22ef01c10684ec2419848bcf6b030b653864f7282a8f6f3 WatchSource:0}: Error finding container 3000b1d9a4b3cb34a22ef01c10684ec2419848bcf6b030b653864f7282a8f6f3: Status 404 returned error can't find the container with id 3000b1d9a4b3cb34a22ef01c10684ec2419848bcf6b030b653864f7282a8f6f3
Sep 11 04:55:18.708540 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:18.708502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" event={"ID":"00614f13-7a64-4725-a20d-7bb35825243b","Type":"ContainerStarted","Data":"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"}
Sep 11 04:55:18.708718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:18.708549    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" event={"ID":"00614f13-7a64-4725-a20d-7bb35825243b","Type":"ContainerStarted","Data":"3000b1d9a4b3cb34a22ef01c10684ec2419848bcf6b030b653864f7282a8f6f3"}
Sep 11 04:55:18.723729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:18.723690    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" podStartSLOduration=1.723675227 podStartE2EDuration="1.723675227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:55:18.722323253 +0000 UTC m=+913.013797689" watchObservedRunningTime="2026-09-11 04:55:18.723675227 +0000 UTC m=+913.015149664"
Sep 11 04:55:24.461873 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.461842    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:24.578932 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.578900    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"]
Sep 11 04:55:24.579346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.579329    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="efae565f-bd7c-4513-8410-b886dc6ad466" containerName="istio-proxy"
Sep 11 04:55:24.582258 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.582238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:24.584542 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.584526    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:55:24.584752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.584734    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:55:24.589899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.589877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"]
Sep 11 04:55:24.682054 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.682031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb\" (UID: \"dde38840-ea78-4eae-8d84-0c0724236ebc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:24.783167 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.783111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb\" (UID: \"dde38840-ea78-4eae-8d84-0c0724236ebc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:24.783698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.783681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb\" (UID: \"dde38840-ea78-4eae-8d84-0c0724236ebc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:24.894795 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:24.894772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:25.160735 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:25.160707    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"]
Sep 11 04:55:25.162520 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:55:25.162497    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde38840_ea78_4eae_8d84_0c0724236ebc.slice/crio-e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e WatchSource:0}: Error finding container e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e: Status 404 returned error can't find the container with id e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e
Sep 11 04:55:25.746962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:25.746921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb" event={"ID":"dde38840-ea78-4eae-8d84-0c0724236ebc","Type":"ContainerStarted","Data":"2fb8c4016de5d9b5f11ce3bc8b62177464409f856b79f8dff93eb1655d858599"}
Sep 11 04:55:25.747319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:25.747007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb" event={"ID":"dde38840-ea78-4eae-8d84-0c0724236ebc","Type":"ContainerStarted","Data":"e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e"}
Sep 11 04:55:25.763472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:25.763428    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb" podStartSLOduration=1.763413903 podStartE2EDuration="1.763413903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:55:25.761108368 +0000 UTC m=+920.052582805" watchObservedRunningTime="2026-09-11 04:55:25.763413903 +0000 UTC m=+920.054888346"
Sep 11 04:55:26.753241 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:26.753206    2581 generic.go:356] "Generic (PLEG): container finished" podID="dde38840-ea78-4eae-8d84-0c0724236ebc" containerID="2fb8c4016de5d9b5f11ce3bc8b62177464409f856b79f8dff93eb1655d858599" exitCode=0
Sep 11 04:55:26.753623 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:26.753289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb" event={"ID":"dde38840-ea78-4eae-8d84-0c0724236ebc","Type":"ContainerDied","Data":"2fb8c4016de5d9b5f11ce3bc8b62177464409f856b79f8dff93eb1655d858599"}
Sep 11 04:55:27.883607 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:27.883567    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:28.009927 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.009856    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca\") pod \"dde38840-ea78-4eae-8d84-0c0724236ebc\" (UID: \"dde38840-ea78-4eae-8d84-0c0724236ebc\") "
Sep 11 04:55:28.010167 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.010145    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dde38840-ea78-4eae-8d84-0c0724236ebc" (UID: "dde38840-ea78-4eae-8d84-0c0724236ebc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:28.071222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.071197    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:28.111145 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.111126    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde38840-ea78-4eae-8d84-0c0724236ebc-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:28.765636 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.765611    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"
Sep 11 04:55:28.765803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.765614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb" event={"ID":"dde38840-ea78-4eae-8d84-0c0724236ebc","Type":"ContainerDied","Data":"e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e"}
Sep 11 04:55:28.765803 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:28.765719    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b9bbbebc653736923748c0998f758c38eccc76f504e13f04b322aef223762e"
Sep 11 04:55:35.184486 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:35.184453    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"]
Sep 11 04:55:35.193275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:35.193248    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-eefe49-lrfvb"]
Sep 11 04:55:36.192790 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:36.192757    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde38840-ea78-4eae-8d84-0c0724236ebc" path="/var/lib/kubelet/pods/dde38840-ea78-4eae-8d84-0c0724236ebc/volumes"
Sep 11 04:55:39.481240 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.481180    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" podUID="00614f13-7a64-4725-a20d-7bb35825243b" containerName="main" containerID="cri-o://ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1" gracePeriod=45
Sep 11 04:55:39.728258 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.728239    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:39.826252 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.826177    2581 generic.go:356] "Generic (PLEG): container finished" podID="00614f13-7a64-4725-a20d-7bb35825243b" containerID="ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1" exitCode=0
Sep 11 04:55:39.826252 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.826212    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"
Sep 11 04:55:39.826420 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.826260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" event={"ID":"00614f13-7a64-4725-a20d-7bb35825243b","Type":"ContainerDied","Data":"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"}
Sep 11 04:55:39.826420 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.826299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb" event={"ID":"00614f13-7a64-4725-a20d-7bb35825243b","Type":"ContainerDied","Data":"3000b1d9a4b3cb34a22ef01c10684ec2419848bcf6b030b653864f7282a8f6f3"}
Sep 11 04:55:39.826420 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.826318    2581 scope.go:117] "RemoveContainer" containerID="ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"
Sep 11 04:55:39.834900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.834886    2581 scope.go:117] "RemoveContainer" containerID="ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"
Sep 11 04:55:39.835175 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:55:39.835157    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1\": container with ID starting with ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1 not found: ID does not exist" containerID="ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"
Sep 11 04:55:39.835222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.835185    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1"} err="failed to get container status \"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1\": rpc error: code = NotFound desc = could not find container \"ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1\": container with ID starting with ab0ebc71c967c17c4b99c7eb08e023a49ad9bf1ebced2fc8f8d59d82d4c4e3d1 not found: ID does not exist"
Sep 11 04:55:39.910444 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ccm9\" (UniqueName: \"kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910546 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910696 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910642    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910696 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910673    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910801 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache\") pod \"00614f13-7a64-4725-a20d-7bb35825243b\" (UID: \"00614f13-7a64-4725-a20d-7bb35825243b\") "
Sep 11 04:55:39.910801 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910708    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home" (OuterVolumeSpecName: "home") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:39.910939 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.910917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:39.911076 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.911059    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:39.911138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.911086    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:39.911138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.911055    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache" (OuterVolumeSpecName: "model-cache") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:39.912830 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.912806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm" (OuterVolumeSpecName: "dshm") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:55:39.912912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.912865    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:55:39.912912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:39.912877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9" (OuterVolumeSpecName: "kube-api-access-4ccm9") pod "00614f13-7a64-4725-a20d-7bb35825243b" (UID: "00614f13-7a64-4725-a20d-7bb35825243b"). InnerVolumeSpecName "kube-api-access-4ccm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:40.011436 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.011412    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ccm9\" (UniqueName: \"kubernetes.io/projected/00614f13-7a64-4725-a20d-7bb35825243b-kube-api-access-4ccm9\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:40.011436 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.011433    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:40.011567 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.011446    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00614f13-7a64-4725-a20d-7bb35825243b-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:40.011567 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.011459    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00614f13-7a64-4725-a20d-7bb35825243b-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:55:40.146239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.146215    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:40.150127 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.150105    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-eefe49/explicit-model-eefe49-kserve-6848d9cc77-jn5gb"]
Sep 11 04:55:40.191896 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:40.191876    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00614f13-7a64-4725-a20d-7bb35825243b" path="/var/lib/kubelet/pods/00614f13-7a64-4725-a20d-7bb35825243b/volumes"
Sep 11 04:55:42.118761 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.118730    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:55:42.119236 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119216    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00614f13-7a64-4725-a20d-7bb35825243b" containerName="main"
Sep 11 04:55:42.119322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119240    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00614f13-7a64-4725-a20d-7bb35825243b" containerName="main"
Sep 11 04:55:42.119322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119256    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde38840-ea78-4eae-8d84-0c0724236ebc" containerName="revoke-keys"
Sep 11 04:55:42.119322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119264    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde38840-ea78-4eae-8d84-0c0724236ebc" containerName="revoke-keys"
Sep 11 04:55:42.119477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119373    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00614f13-7a64-4725-a20d-7bb35825243b" containerName="main"
Sep 11 04:55:42.119477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.119388    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde38840-ea78-4eae-8d84-0c0724236ebc" containerName="revoke-keys"
Sep 11 04:55:42.123690 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.123658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.125756 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.125736    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-6bc1bc-openshift-default-dockercfg-xn75f\""
Sep 11 04:55:42.129494 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.129473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.129716 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.129693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.129927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.129979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.130028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.130055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.130114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bghrw\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.130145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.130560 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.130208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.133469 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.133446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:55:42.230527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bghrw\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.230874 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.230801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.231105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.231080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.231164 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.231128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.231219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.231189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.231320 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.231297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.231426 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.231405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.233682 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.233661    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.233888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.233870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.238030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.238010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.238111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.238040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bghrw\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw\") pod \"e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") " pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.434020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.433935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:42.708955 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.708928    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:55:42.710128 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:55:42.710101    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709d5975_a23d_4438_88f4_927289b50373.slice/crio-9723783519cf0862013dd12224d76ab6f19aa57a396c57fb303b980f299d6fc0 WatchSource:0}: Error finding container 9723783519cf0862013dd12224d76ab6f19aa57a396c57fb303b980f299d6fc0: Status 404 returned error can't find the container with id 9723783519cf0862013dd12224d76ab6f19aa57a396c57fb303b980f299d6fc0
Sep 11 04:55:42.713101 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.712715    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:55:42.713101 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.712794    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:55:42.713101 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.712859    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:55:42.847039 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.846998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" event={"ID":"709d5975-a23d-4438-88f4-927289b50373","Type":"ContainerStarted","Data":"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"}
Sep 11 04:55:42.847202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.847067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" event={"ID":"709d5975-a23d-4438-88f4-927289b50373","Type":"ContainerStarted","Data":"9723783519cf0862013dd12224d76ab6f19aa57a396c57fb303b980f299d6fc0"}
Sep 11 04:55:42.865477 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:42.865429    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" podStartSLOduration=0.865413914 podStartE2EDuration="865.413914ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:55:42.864172079 +0000 UTC m=+937.155646522" watchObservedRunningTime="2026-09-11 04:55:42.865413914 +0000 UTC m=+937.156888351"
Sep 11 04:55:43.434682 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:43.434642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:43.439527 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:43.439504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:43.852052 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:43.852028    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:43.853053 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:43.853032    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:55:55.446346 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.446318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:55:55.449486 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.449471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.451530 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.451509    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6bc1bc\"/\"kube-root-ca.crt\""
Sep 11 04:55:55.451698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.451510    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6bc1bc\"/\"default-dockercfg-c65xn\""
Sep 11 04:55:55.452202 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.452182    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6bc1bc\"/\"openshift-service-ca.crt\""
Sep 11 04:55:55.452307 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.452255    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6bc1bc\"/\"noten-model-6bc1bc-kserve-self-signed-certs\""
Sep 11 04:55:55.458421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.458400    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:55:55.538569 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.538752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcmgt\" (UniqueName: \"kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.538752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.538752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538712    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.538752 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.538890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.538766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639430 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcmgt\" (UniqueName: \"kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639606 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.639965 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.639937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.640025 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.640010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.641944 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.641916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.642173 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.642155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.647841 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.647820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcmgt\" (UniqueName: \"kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt\") pod \"noten-model-6bc1bc-kserve-86d6848dcf-clmnp\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") " pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:55.760178 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:55.760089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:55:56.031041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:56.031012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:55:56.032278 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:55:56.032255    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5d8dc9a_ad81_4947_a309_0a5e478b4cf2.slice/crio-5501232b700ad514303869f1eff2d78c13b9765e0a218e889df0d5b30e225a98 WatchSource:0}: Error finding container 5501232b700ad514303869f1eff2d78c13b9765e0a218e889df0d5b30e225a98: Status 404 returned error can't find the container with id 5501232b700ad514303869f1eff2d78c13b9765e0a218e889df0d5b30e225a98
Sep 11 04:55:56.930825 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:56.930791    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" event={"ID":"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2","Type":"ContainerStarted","Data":"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"}
Sep 11 04:55:56.931193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:56.930833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" event={"ID":"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2","Type":"ContainerStarted","Data":"5501232b700ad514303869f1eff2d78c13b9765e0a218e889df0d5b30e225a98"}
Sep 11 04:55:56.947742 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:55:56.947693    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" podStartSLOduration=1.94767908 podStartE2EDuration="1.94767908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:55:56.945454068 +0000 UTC m=+951.236928504" watchObservedRunningTime="2026-09-11 04:55:56.94767908 +0000 UTC m=+951.239153517"
Sep 11 04:56:04.490118 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:04.490088    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:56:04.573407 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:04.573370    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:56:04.573656 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:04.573631    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" podUID="709d5975-a23d-4438-88f4-927289b50373" containerName="istio-proxy" containerID="cri-o://4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112" gracePeriod=30
Sep 11 04:56:05.761179 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:05.761144    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:56:09.823062 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.823035    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:56:09.966470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966651 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966651 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966651 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966533    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bghrw\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.966845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:09.966845 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966824    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data" (OuterVolumeSpecName: "istio-data") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:09.967105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966880    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.967105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966884    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:09.967105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.966907    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket\") pod \"709d5975-a23d-4438-88f4-927289b50373\" (UID: \"709d5975-a23d-4438-88f4-927289b50373\") "
Sep 11 04:56:09.967263 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967112    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:09.967263 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967133    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:09.967263 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967235    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:09.967263 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967253    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:09.967478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967266    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:09.967478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967280    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709d5975-a23d-4438-88f4-927289b50373-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:09.967478 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.967291    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:09.968945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.968917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:09.969092 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.969070    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token" (OuterVolumeSpecName: "istio-token") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:09.969160 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.969086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw" (OuterVolumeSpecName: "kube-api-access-bghrw") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "kube-api-access-bghrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:09.969219 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:09.969153    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "709d5975-a23d-4438-88f4-927289b50373" (UID: "709d5975-a23d-4438-88f4-927289b50373"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:56:10.005416 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.005387    2581 generic.go:356] "Generic (PLEG): container finished" podID="709d5975-a23d-4438-88f4-927289b50373" containerID="4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112" exitCode=0
Sep 11 04:56:10.005543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.005435    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"
Sep 11 04:56:10.005543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.005477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" event={"ID":"709d5975-a23d-4438-88f4-927289b50373","Type":"ContainerDied","Data":"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"}
Sep 11 04:56:10.005633 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.005558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5" event={"ID":"709d5975-a23d-4438-88f4-927289b50373","Type":"ContainerDied","Data":"9723783519cf0862013dd12224d76ab6f19aa57a396c57fb303b980f299d6fc0"}
Sep 11 04:56:10.005633 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.005598    2581 scope.go:117] "RemoveContainer" containerID="4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"
Sep 11 04:56:10.015003 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.014989    2581 scope.go:117] "RemoveContainer" containerID="4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"
Sep 11 04:56:10.015266 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:56:10.015251    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112\": container with ID starting with 4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112 not found: ID does not exist" containerID="4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"
Sep 11 04:56:10.015312 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.015274    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112"} err="failed to get container status \"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112\": rpc error: code = NotFound desc = could not find container \"4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112\": container with ID starting with 4842685c93e3d2a7152174ebc8b023f3b3a4564e1d4f38abad05b2a3e054f112 not found: ID does not exist"
Sep 11 04:56:10.025869 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.025849    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:56:10.029730 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.029711    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-6bc1bc-openshift-default-6fbdff66-888f5"]
Sep 11 04:56:10.067945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.067921    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709d5975-a23d-4438-88f4-927289b50373-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:10.067945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.067942    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bghrw\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-kube-api-access-bghrw\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:10.068059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.067953    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709d5975-a23d-4438-88f4-927289b50373-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:10.068059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.067961    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709d5975-a23d-4438-88f4-927289b50373-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:10.193732 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:10.193708    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709d5975-a23d-4438-88f4-927289b50373" path="/var/lib/kubelet/pods/709d5975-a23d-4438-88f4-927289b50373/volumes"
Sep 11 04:56:18.703697 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:18.703667    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:56:19.509493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.509440    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" podUID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" containerName="main" containerID="cri-o://5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc" gracePeriod=45
Sep 11 04:56:19.744476 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.744444    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:56:19.758689 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.758669    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:56:19.852456 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852456 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852434    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcmgt\" (UniqueName: \"kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852566    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852637    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm\") pod \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\" (UID: \"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2\") "
Sep 11 04:56:19.852729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852674    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home" (OuterVolumeSpecName: "home") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:19.853002 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852767    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:19.853002 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.852907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:19.853091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.853011    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:19.853091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.853024    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:19.853091 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.853035    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:19.854846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.854824    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:56:19.854932 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.854893    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm" (OuterVolumeSpecName: "dshm") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:19.854932 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.854911    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt" (OuterVolumeSpecName: "kube-api-access-dcmgt") pod "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" (UID: "d5d8dc9a-ad81-4947-a309-0a5e478b4cf2"). InnerVolumeSpecName "kube-api-access-dcmgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:19.953875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.953846    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:19.953875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.953868    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:19.953875 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:19.953880    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcmgt\" (UniqueName: \"kubernetes.io/projected/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2-kube-api-access-dcmgt\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:20.061926 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.061897    2581 generic.go:356] "Generic (PLEG): container finished" podID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" containerID="5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc" exitCode=0
Sep 11 04:56:20.062088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.061941    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"
Sep 11 04:56:20.062088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.061980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" event={"ID":"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2","Type":"ContainerDied","Data":"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"}
Sep 11 04:56:20.062088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.062022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp" event={"ID":"d5d8dc9a-ad81-4947-a309-0a5e478b4cf2","Type":"ContainerDied","Data":"5501232b700ad514303869f1eff2d78c13b9765e0a218e889df0d5b30e225a98"}
Sep 11 04:56:20.062088 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.062040    2581 scope.go:117] "RemoveContainer" containerID="5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"
Sep 11 04:56:20.071439 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.071423    2581 scope.go:117] "RemoveContainer" containerID="5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"
Sep 11 04:56:20.071708 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:56:20.071692    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc\": container with ID starting with 5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc not found: ID does not exist" containerID="5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"
Sep 11 04:56:20.071757 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.071717    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc"} err="failed to get container status \"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc\": rpc error: code = NotFound desc = could not find container \"5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc\": container with ID starting with 5a0038262ba5a79da6e5dc4cf4067742596b68bd2137babe74be853292eef1fc not found: ID does not exist"
Sep 11 04:56:20.081771 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.081750    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:56:20.084682 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.084664    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-6bc1bc/noten-model-6bc1bc-kserve-86d6848dcf-clmnp"]
Sep 11 04:56:20.192292 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:20.192242    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" path="/var/lib/kubelet/pods/d5d8dc9a-ad81-4947-a309-0a5e478b4cf2/volumes"
Sep 11 04:56:33.722898 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.722804    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" podUID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" containerName="main" containerID="cri-o://5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549" gracePeriod=45
Sep 11 04:56:33.966695 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.966671    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:56:33.970558 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970539    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbz8j\" (UniqueName: \"kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970615    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970637    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970699    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970861 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home\") pod \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\" (UID: \"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba\") "
Sep 11 04:56:33.970979 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.970877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:33.971086 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.971066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home" (OuterVolumeSpecName: "home") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:33.971150 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.971105    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:33.971363 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.971234    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:33.973421 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.973360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm" (OuterVolumeSpecName: "dshm") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:33.973525 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.973446    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j" (OuterVolumeSpecName: "kube-api-access-rbz8j") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "kube-api-access-rbz8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:33.974100 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:33.974049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" (UID: "85b1e221-bb5d-44b5-a6e4-94d0b137a8ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:56:34.071751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.071719    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:34.071751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.071758    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:34.071950 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.071767    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbz8j\" (UniqueName: \"kubernetes.io/projected/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-kube-api-access-rbz8j\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:34.071950 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.071776    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:34.071950 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.071786    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:34.136798 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.136767    2581 generic.go:356] "Generic (PLEG): container finished" podID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" containerID="5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549" exitCode=0
Sep 11 04:56:34.136940 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.136815    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"
Sep 11 04:56:34.136940 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.136855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" event={"ID":"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba","Type":"ContainerDied","Data":"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"}
Sep 11 04:56:34.136940 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.136899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66" event={"ID":"85b1e221-bb5d-44b5-a6e4-94d0b137a8ba","Type":"ContainerDied","Data":"860ec4d44774533cafc7fac67da1599b1623deaf9fa9835e780e51190790b977"}
Sep 11 04:56:34.136940 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.136916    2581 scope.go:117] "RemoveContainer" containerID="5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"
Sep 11 04:56:34.146128 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.146111    2581 scope.go:117] "RemoveContainer" containerID="5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"
Sep 11 04:56:34.146396 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:56:34.146374    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549\": container with ID starting with 5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549 not found: ID does not exist" containerID="5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"
Sep 11 04:56:34.146459 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.146408    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549"} err="failed to get container status \"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549\": rpc error: code = NotFound desc = could not find container \"5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549\": container with ID starting with 5f9cb856dafa24dcba19a5148ea4cb64789abc61022045bc883870c2ad594549 not found: ID does not exist"
Sep 11 04:56:34.158209 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.158189    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:56:34.162144 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.162125    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-831a1c/test-model-831a1c-kserve-794854cbbc-52m66"]
Sep 11 04:56:34.191621 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.191602    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" path="/var/lib/kubelet/pods/85b1e221-bb5d-44b5-a6e4-94d0b137a8ba/volumes"
Sep 11 04:56:34.764994 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:34.764960    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" podUID="c63042eb-3814-4711-8f11-e531d1506297" containerName="main" containerID="cri-o://dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e" gracePeriod=45
Sep 11 04:56:35.009574 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.009545    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:56:35.080739 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh6p4\" (UniqueName: \"kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.080739 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.080898 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080752    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.080898 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080784    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.080898 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.080898 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.080879    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs\") pod \"c63042eb-3814-4711-8f11-e531d1506297\" (UID: \"c63042eb-3814-4711-8f11-e531d1506297\") "
Sep 11 04:56:35.081113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:35.081113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081038    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache" (OuterVolumeSpecName: "model-cache") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:35.081113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081082    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home" (OuterVolumeSpecName: "home") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:35.081291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081156    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:35.081291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081174    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:35.081291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.081183    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:35.083120 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.083096    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm" (OuterVolumeSpecName: "dshm") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:35.083204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.083129    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:56:35.083204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.083177    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4" (OuterVolumeSpecName: "kube-api-access-lh6p4") pod "c63042eb-3814-4711-8f11-e531d1506297" (UID: "c63042eb-3814-4711-8f11-e531d1506297"). InnerVolumeSpecName "kube-api-access-lh6p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:35.144189 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.144161    2581 generic.go:356] "Generic (PLEG): container finished" podID="c63042eb-3814-4711-8f11-e531d1506297" containerID="dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e" exitCode=0
Sep 11 04:56:35.144311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.144197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" event={"ID":"c63042eb-3814-4711-8f11-e531d1506297","Type":"ContainerDied","Data":"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"}
Sep 11 04:56:35.144311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.144204    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"
Sep 11 04:56:35.144311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.144228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db" event={"ID":"c63042eb-3814-4711-8f11-e531d1506297","Type":"ContainerDied","Data":"fde40cd5ae3886e138415e4395740d500cb02167a5d734430b9b2ff1a0f01f5a"}
Sep 11 04:56:35.144311 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.144249    2581 scope.go:117] "RemoveContainer" containerID="dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"
Sep 11 04:56:35.153142 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.153122    2581 scope.go:117] "RemoveContainer" containerID="dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"
Sep 11 04:56:35.153429 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:56:35.153411    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e\": container with ID starting with dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e not found: ID does not exist" containerID="dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"
Sep 11 04:56:35.153475 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.153439    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e"} err="failed to get container status \"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e\": rpc error: code = NotFound desc = could not find container \"dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e\": container with ID starting with dede983301044781656d718340de7a75078fe0c251e674c6a926db46c0c6449e not found: ID does not exist"
Sep 11 04:56:35.164173 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.164152    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:56:35.167849 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.167830    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7bdb3c/test-model-7bdb3c-kserve-7dbfbdbd67-gm8db"]
Sep 11 04:56:35.181981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.181964    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c63042eb-3814-4711-8f11-e531d1506297-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:35.182056 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.181983    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c63042eb-3814-4711-8f11-e531d1506297-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:35.182056 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:35.181993    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh6p4\" (UniqueName: \"kubernetes.io/projected/c63042eb-3814-4711-8f11-e531d1506297-kube-api-access-lh6p4\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:36.192368 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:36.192335    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63042eb-3814-4711-8f11-e531d1506297" path="/var/lib/kubelet/pods/c63042eb-3814-4711-8f11-e531d1506297/volumes"
Sep 11 04:56:40.399858 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.399822    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400167    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63042eb-3814-4711-8f11-e531d1506297" containerName="main"
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400179    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63042eb-3814-4711-8f11-e531d1506297" containerName="main"
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400195    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" containerName="main"
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400200    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" containerName="main"
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400213    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709d5975-a23d-4438-88f4-927289b50373" containerName="istio-proxy"
Sep 11 04:56:40.400216 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400219    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="709d5975-a23d-4438-88f4-927289b50373" containerName="istio-proxy"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400225    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" containerName="main"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400231    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" containerName="main"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400293    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b1e221-bb5d-44b5-a6e4-94d0b137a8ba" containerName="main"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400301    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="709d5975-a23d-4438-88f4-927289b50373" containerName="istio-proxy"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400306    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d8dc9a-ad81-4947-a309-0a5e478b4cf2" containerName="main"
Sep 11 04:56:40.400402 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.400312    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63042eb-3814-4711-8f11-e531d1506297" containerName="main"
Sep 11 04:56:40.404556 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.404535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.406678 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.406658    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ca9fd5b9-openshift-default-dockercfg-qkx8j\""
Sep 11 04:56:40.416804 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.416783    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:56:40.424561 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhkg8\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424824 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424824 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.424890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.424880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525247 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525247 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhkg8\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525492 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.525846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.526030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.526030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.525910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.526200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.526179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.527859 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.527838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.527954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.527939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.533506 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.533481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.533960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.533941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhkg8\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8\") pod \"e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") " pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.715405 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.715383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:40.991291 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.991266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:56:40.992905 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:56:40.992879    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b09d17f_8aa3_4aaa_8e70_1b5c7dd3bcb3.slice/crio-d37e588b2052466f920fd57381da50ed3dea7e8c053486d6912ee63303f4b0d4 WatchSource:0}: Error finding container d37e588b2052466f920fd57381da50ed3dea7e8c053486d6912ee63303f4b0d4: Status 404 returned error can't find the container with id d37e588b2052466f920fd57381da50ed3dea7e8c053486d6912ee63303f4b0d4
Sep 11 04:56:40.995253 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.995223    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:40.995334 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.995284    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:40.995334 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:40.995311    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:41.181084 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:41.181045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerStarted","Data":"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"}
Sep 11 04:56:41.181275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:41.181114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerStarted","Data":"d37e588b2052466f920fd57381da50ed3dea7e8c053486d6912ee63303f4b0d4"}
Sep 11 04:56:41.204877 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:41.204825    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podStartSLOduration=1.204804632 podStartE2EDuration="1.204804632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:56:41.202026188 +0000 UTC m=+995.493500626" watchObservedRunningTime="2026-09-11 04:56:41.204804632 +0000 UTC m=+995.496279070"
Sep 11 04:56:41.715856 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:41.715812    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:42.720413 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:42.720385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:43.193517 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:43.193440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:43.194416 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:43.194392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:54.156981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:54.156949    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:56:54.157458 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:54.157177    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" containerID="cri-o://162786e2b8ea0727611489cb5d672d76875fe030f393fd332644f109d408fa4a" gracePeriod=30
Sep 11 04:56:54.987451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:54.987368    2581 patch_prober.go:28] interesting pod/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 04:56:54.987451 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:54.987431    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 04:56:58.195343 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.195312    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:56:58.195758 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.195373    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:56:58.290345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.290315    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerID="f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15" exitCode=0
Sep 11 04:56:58.290505 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.290384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerDied","Data":"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"}
Sep 11 04:56:58.290790 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.290776    2581 scope.go:117] "RemoveContainer" containerID="f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"
Sep 11 04:56:58.291775 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.291750    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:58.291846 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.291814    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:58.291892 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:58.291844    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:56:59.298817 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.298788    2581 generic.go:356] "Generic (PLEG): container finished" podID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerID="162786e2b8ea0727611489cb5d672d76875fe030f393fd332644f109d408fa4a" exitCode=0
Sep 11 04:56:59.299222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.298868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerDied","Data":"162786e2b8ea0727611489cb5d672d76875fe030f393fd332644f109d408fa4a"}
Sep 11 04:56:59.299222 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.298941    2581 scope.go:117] "RemoveContainer" containerID="a7c1da84d46ee3c304496a89a21b2a467b852bd94504095ab25630288687d3ed"
Sep 11 04:56:59.301170 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.301144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerStarted","Data":"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"}
Sep 11 04:56:59.411041 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.411020    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:56:59.505447 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505482    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqg9c\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505543    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505573    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505654    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505716    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505745    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy\") pod \"91f4bf76-0202-468c-aa53-1de6d690eb82\" (UID: \"91f4bf76-0202-468c-aa53-1de6d690eb82\") "
Sep 11 04:56:59.505935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505801    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data" (OuterVolumeSpecName: "istio-data") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:59.505935 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.505918    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:59.506194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.506068    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:59.506194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.506084    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.506194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.506105    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.506194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.506141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:59.506194 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.506184    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:59.507974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.507943    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:56:59.508124 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.508102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c" (OuterVolumeSpecName: "kube-api-access-rqg9c") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "kube-api-access-rqg9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:59.508376 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.508354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:56:59.508431 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.508401    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token" (OuterVolumeSpecName: "istio-token") pod "91f4bf76-0202-468c-aa53-1de6d690eb82" (UID: "91f4bf76-0202-468c-aa53-1de6d690eb82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:59.607274 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607242    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607274 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607283    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f4bf76-0202-468c-aa53-1de6d690eb82-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607293    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607301    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607310    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqg9c\" (UniqueName: \"kubernetes.io/projected/91f4bf76-0202-468c-aa53-1de6d690eb82-kube-api-access-rqg9c\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607317    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f4bf76-0202-468c-aa53-1de6d690eb82-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.607470 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.607325    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f4bf76-0202-468c-aa53-1de6d690eb82-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:56:59.715960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.715931    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:56:59.716180 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.716157    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:56:59.716288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:56:59.716227    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:57:00.313563 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.313530    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"
Sep 11 04:57:00.314034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.313562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf" event={"ID":"91f4bf76-0202-468c-aa53-1de6d690eb82","Type":"ContainerDied","Data":"b4d3f29368b2988a3c4c1c01aa42d778d7c5a4d1b9d1a759773e802f4bb3936c"}
Sep 11 04:57:00.314034 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.313663    2581 scope.go:117] "RemoveContainer" containerID="162786e2b8ea0727611489cb5d672d76875fe030f393fd332644f109d408fa4a"
Sep 11 04:57:00.348676 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.348633    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:57:00.349342 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.349310    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7bdb3c-openshift-default-5c44f5b6c4-gfksf"]
Sep 11 04:57:00.716348 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.716317    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:57:00.716488 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:00.716374    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:57:01.716591 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:01.716554    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:57:01.717014 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:01.716634    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:57:02.192512 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:02.192418    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" path="/var/lib/kubelet/pods/91f4bf76-0202-468c-aa53-1de6d690eb82/volumes"
Sep 11 04:57:02.715890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:02.715856    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:57:02.716039 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:02.715920    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:57:03.716671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:03.716639    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 11 04:57:03.717038 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:03.716693    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 11 04:57:04.720328 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:04.720294    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:57:04.720854 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:04.720591    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:57:04.721142 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:04.721128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:57:15.254211 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254178    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"]
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254514    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254529    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254540    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254545    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254619    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.254639 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.254636    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f4bf76-0202-468c-aa53-1de6d690eb82" containerName="istio-proxy"
Sep 11 04:57:15.259070 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.259053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:15.261162 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.261139    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:57:15.261288 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.261238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:57:15.265060 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.265035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"]
Sep 11 04:57:15.447496 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.447456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84\" (UID: \"5b7bb5ad-ac41-4e01-846c-8472c14556d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:15.548631 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.548517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84\" (UID: \"5b7bb5ad-ac41-4e01-846c-8472c14556d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:15.549181 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.549157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84\" (UID: \"5b7bb5ad-ac41-4e01-846c-8472c14556d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:15.571600 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.571558    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:15.839941 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:15.839922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"]
Sep 11 04:57:15.842160 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:57:15.842129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7bb5ad_ac41_4e01_846c_8472c14556d6.slice/crio-5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952 WatchSource:0}: Error finding container 5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952: Status 404 returned error can't find the container with id 5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952
Sep 11 04:57:16.405543 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:16.405501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84" event={"ID":"5b7bb5ad-ac41-4e01-846c-8472c14556d6","Type":"ContainerStarted","Data":"1a48ee3e1ac8ad9dc396da0b69a7bf5f01c82e5589f4bcbdcdcc55d323fd9257"}
Sep 11 04:57:16.405945 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:16.405563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84" event={"ID":"5b7bb5ad-ac41-4e01-846c-8472c14556d6","Type":"ContainerStarted","Data":"5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952"}
Sep 11 04:57:16.421472 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:16.421426    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84" podStartSLOduration=1.421412265 podStartE2EDuration="1.421412265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:16.419227021 +0000 UTC m=+1030.710701473" watchObservedRunningTime="2026-09-11 04:57:16.421412265 +0000 UTC m=+1030.712886701"
Sep 11 04:57:17.412046 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:17.412009    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b7bb5ad-ac41-4e01-846c-8472c14556d6" containerID="1a48ee3e1ac8ad9dc396da0b69a7bf5f01c82e5589f4bcbdcdcc55d323fd9257" exitCode=0
Sep 11 04:57:17.412433 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:17.412071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84" event={"ID":"5b7bb5ad-ac41-4e01-846c-8472c14556d6","Type":"ContainerDied","Data":"1a48ee3e1ac8ad9dc396da0b69a7bf5f01c82e5589f4bcbdcdcc55d323fd9257"}
Sep 11 04:57:18.550164 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:18.550142    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:18.677912 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:18.677831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca\") pod \"5b7bb5ad-ac41-4e01-846c-8472c14556d6\" (UID: \"5b7bb5ad-ac41-4e01-846c-8472c14556d6\") "
Sep 11 04:57:18.678145 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:18.678125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b7bb5ad-ac41-4e01-846c-8472c14556d6" (UID: "5b7bb5ad-ac41-4e01-846c-8472c14556d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:57:18.778685 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:18.778648    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b7bb5ad-ac41-4e01-846c-8472c14556d6-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:19.430252 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:19.430224    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"
Sep 11 04:57:19.430398 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:19.430222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84" event={"ID":"5b7bb5ad-ac41-4e01-846c-8472c14556d6","Type":"ContainerDied","Data":"5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952"}
Sep 11 04:57:19.430398 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:19.430349    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f6c9e0fcf75193b145f03c969d2edd757a72a65c030f69867add8479824b952"
Sep 11 04:57:23.314295 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.314249    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 04:57:23.314829 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.314611    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7bb5ad-ac41-4e01-846c-8472c14556d6" containerName="revoke-keys"
Sep 11 04:57:23.314829 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.314625    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7bb5ad-ac41-4e01-846c-8472c14556d6" containerName="revoke-keys"
Sep 11 04:57:23.314829 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.314689    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7bb5ad-ac41-4e01-846c-8472c14556d6" containerName="revoke-keys"
Sep 11 04:57:23.317644 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.317625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.319778 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.319760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-16b799-openshift-default-dockercfg-72srn\""
Sep 11 04:57:23.328118 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.328100    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 04:57:23.415496 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqdcp\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415708 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.415897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.415890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516315 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516315 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516602 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516602 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516602 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqdcp\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.516777 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.517071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.517071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.516701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.517071 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.517040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.517200 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.517108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.518809 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.518787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.519078 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.519062    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.524462 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.524441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.524651 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.524634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqdcp\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp\") pod \"e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") " pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.628024 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.627949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:23.936993 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.936873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 04:57:23.940983 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:57:23.940957    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b009752_948a_4724_9a61_45af15c8b7f7.slice/crio-a9e5efe4064f8b59c349345108771b6435c130a707b1f5fdfb13cace9e61e17a WatchSource:0}: Error finding container a9e5efe4064f8b59c349345108771b6435c130a707b1f5fdfb13cace9e61e17a: Status 404 returned error can't find the container with id a9e5efe4064f8b59c349345108771b6435c130a707b1f5fdfb13cace9e61e17a
Sep 11 04:57:23.943597 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.943552    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:23.943703 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.943657    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:23.943768 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:23.943702    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:24.458734 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:24.458699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerStarted","Data":"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"}
Sep 11 04:57:24.459123 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:24.458755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerStarted","Data":"a9e5efe4064f8b59c349345108771b6435c130a707b1f5fdfb13cace9e61e17a"}
Sep 11 04:57:24.479111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:24.479063    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podStartSLOduration=1.47904831 podStartE2EDuration="1.47904831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:24.477200467 +0000 UTC m=+1038.768674904" watchObservedRunningTime="2026-09-11 04:57:24.47904831 +0000 UTC m=+1038.770522746"
Sep 11 04:57:24.628941 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:24.628909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:25.633809 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:25.633779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:26.099026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:26.098997    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"]
Sep 11 04:57:26.101360 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:26.101337    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ca9fd5b9-vks84"]
Sep 11 04:57:26.193760 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:26.193681    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7bb5ad-ac41-4e01-846c-8472c14556d6" path="/var/lib/kubelet/pods/5b7bb5ad-ac41-4e01-846c-8472c14556d6/volumes"
Sep 11 04:57:26.469974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:26.469945    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:26.471145 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:26.471122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:32.661105 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:32.661075    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:57:32.661593 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:32.661333    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" containerID="cri-o://e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad" gracePeriod=30
Sep 11 04:57:34.723413 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:34.723372    2581 patch_prober.go:28] interesting pod/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 04:57:34.723833 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:34.723450    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 04:57:37.959138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:37.959115    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:57:38.041869 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.041837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042095 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042239 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042226    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042353 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042462 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042450    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042647 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042630    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042769 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042757    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhkg8\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.042893 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042880    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.043006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.042993    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token\") pod \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\" (UID: \"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3\") "
Sep 11 04:57:38.044032 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.044007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:57:38.044384 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.044366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:57:38.045677 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.044956    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:57:38.045677 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.045261    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:57:38.045677 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.045645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:57:38.048996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.048936    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:57:38.049098 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.049050    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:57:38.049165 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.049141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8" (OuterVolumeSpecName: "kube-api-access-qhkg8") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "kube-api-access-qhkg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:57:38.049331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.049306    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" (UID: "9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144164    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144193    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144208    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144222    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144237    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144249    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144262    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhkg8\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-kube-api-access-qhkg8\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144276    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.144308 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.144288    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:57:38.542773 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.542675    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerID="e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad" exitCode=0
Sep 11 04:57:38.542773 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.542739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"
Sep 11 04:57:38.542995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.542779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerDied","Data":"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"}
Sep 11 04:57:38.542995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.542829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7" event={"ID":"9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3","Type":"ContainerDied","Data":"d37e588b2052466f920fd57381da50ed3dea7e8c053486d6912ee63303f4b0d4"}
Sep 11 04:57:38.542995 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.542860    2581 scope.go:117] "RemoveContainer" containerID="e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"
Sep 11 04:57:38.555642 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.555621    2581 scope.go:117] "RemoveContainer" containerID="f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"
Sep 11 04:57:38.561482 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.561462    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:57:38.563601 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.563566    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ca9fd5b9-openshift-default-59f9f6574d-24gx7"]
Sep 11 04:57:38.565318 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.565295    2581 scope.go:117] "RemoveContainer" containerID="e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"
Sep 11 04:57:38.565611 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:57:38.565593    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad\": container with ID starting with e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad not found: ID does not exist" containerID="e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"
Sep 11 04:57:38.565674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.565618    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad"} err="failed to get container status \"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad\": rpc error: code = NotFound desc = could not find container \"e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad\": container with ID starting with e517abf454c92fe94708107d606f8f97fa3d0559842d90b2c33f4e28c7dc0cad not found: ID does not exist"
Sep 11 04:57:38.565674 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.565635    2581 scope.go:117] "RemoveContainer" containerID="f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"
Sep 11 04:57:38.565912 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:57:38.565892    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15\": container with ID starting with f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15 not found: ID does not exist" containerID="f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"
Sep 11 04:57:38.565963 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:38.565923    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15"} err="failed to get container status \"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15\": rpc error: code = NotFound desc = could not find container \"f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15\": container with ID starting with f089976a53f0b038e158dc445e5f4167676b0b8dcafabab888d9fced1a9cbe15 not found: ID does not exist"
Sep 11 04:57:40.197019 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:40.196985    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" path="/var/lib/kubelet/pods/9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3/volumes"
Sep 11 04:57:42.511756 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:42.511715    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:57:42.512138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:42.511793    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:57:43.574659 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.574560    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b009752-948a-4724-9a61-45af15c8b7f7" containerID="cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104" exitCode=0
Sep 11 04:57:43.574659 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.574645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerDied","Data":"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"}
Sep 11 04:57:43.575045 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.575020    2581 scope.go:117] "RemoveContainer" containerID="cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"
Sep 11 04:57:43.576113 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.576089    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:43.576192 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.576158    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:43.576229 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:43.576192    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:44.581864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:44.581834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerStarted","Data":"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"}
Sep 11 04:57:44.628751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:44.628725    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:44.629132 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:44.629107    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 11 04:57:44.629245 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:44.629162    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 11 04:57:45.629199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:45.629166    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 11 04:57:45.629840 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:45.629807    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 11 04:57:46.183937 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.183907    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 04:57:46.184238 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184216    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.184238 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184230    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.184385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184250    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.184385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184255    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.184385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184308    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.184385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.184318    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b09d17f-8aa3-4aaa-8e70-1b5c7dd3bcb3" containerName="istio-proxy"
Sep 11 04:57:46.188915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.188897    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.190899 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.190882    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-f5e6f7-openshift-default-dockercfg-jfnx4\""
Sep 11 04:57:46.197940 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.197922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 04:57:46.336553 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.336518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.336873 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.336850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4spm\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.337075 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.337059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.337277 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.337261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.337900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.337879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.338121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.338103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.338275 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.338260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.338437 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.338423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.338622 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.338606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4spm\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.440169 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.439550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.441644 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.441620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.441795 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.441651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.441891 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.441695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.441977 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.441931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.442336 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.442309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.442965 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.442918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.444713 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.444690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.450608 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.450114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4spm\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.454157 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.454133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token\") pod \"e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") " pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.501366 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.501339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:46.634605 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.632969    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 11 04:57:46.634605 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.633032    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 11 04:57:46.805372 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.805347    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 04:57:46.806719 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:57:46.806699    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ec5a88_1186_4a50_b54b_a611c22eba18.slice/crio-96e73edc309ca81fdfebe67afab3b7b211a14f771ef459473b4ebf41bfbaf524 WatchSource:0}: Error finding container 96e73edc309ca81fdfebe67afab3b7b211a14f771ef459473b4ebf41bfbaf524: Status 404 returned error can't find the container with id 96e73edc309ca81fdfebe67afab3b7b211a14f771ef459473b4ebf41bfbaf524
Sep 11 04:57:46.808869 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.808840    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:46.808962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.808898    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:46.808962 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:46.808923    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:57:47.604514 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:47.604477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerStarted","Data":"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"}
Sep 11 04:57:47.604717 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:47.604536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerStarted","Data":"96e73edc309ca81fdfebe67afab3b7b211a14f771ef459473b4ebf41bfbaf524"}
Sep 11 04:57:47.625562 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:47.625504    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podStartSLOduration=1.625484285 podStartE2EDuration="1.625484285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:47.622143052 +0000 UTC m=+1061.913617488" watchObservedRunningTime="2026-09-11 04:57:47.625484285 +0000 UTC m=+1061.916958724"
Sep 11 04:57:47.628268 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:47.628233    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 11 04:57:47.628377 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:47.628317    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 11 04:57:48.501513 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.501477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:48.506385 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.506366    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:48.609996 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.609966    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:48.610975 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.610961    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:57:48.628328 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.628293    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 11 04:57:48.628501 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:48.628350    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 11 04:57:49.645673 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:49.645641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:49.646143 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:49.646104    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:57:49.647618 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:57:49.647599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 04:58:04.651701 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:04.651664    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:58:04.652114 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:04.651726    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:58:06.713474 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.713442    2581 generic.go:356] "Generic (PLEG): container finished" podID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerID="2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea" exitCode=0
Sep 11 04:58:06.714067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.713504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerDied","Data":"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"}
Sep 11 04:58:06.714067 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.713887    2581 scope.go:117] "RemoveContainer" containerID="2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"
Sep 11 04:58:06.714904 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.714880    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:06.714983 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.714951    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:06.715020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:06.714985    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:07.720922 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:07.720883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerStarted","Data":"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"}
Sep 11 04:58:08.502247 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:08.502211    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:58:08.502422 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:08.502404    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 04:58:08.502468 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:08.502451    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 04:58:09.502728 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:09.502701    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 04:58:09.503098 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:09.502751    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 04:58:10.502349 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:10.502314    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 04:58:10.502533 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:10.502368    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 04:58:11.502345 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:11.502314    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 04:58:11.502746 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:11.502364    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 04:58:12.502051 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:12.502013    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 11 04:58:12.502257 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:12.502087    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 11 04:58:13.505544 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:13.505514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:58:13.506013 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:13.505921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:58:13.506463 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:13.506443    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 04:58:15.046671 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.046639    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 04:58:15.049023 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.049003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.051349 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.051333    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f81bf2-openshift-default-dockercfg-8tlpq\""
Sep 11 04:58:15.062075 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.062051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 04:58:15.093193 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093426 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093426 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn49b\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.093521 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.093505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194572 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.194788 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn49b\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195148 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195148 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.194980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195148 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.195087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195148 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.195099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195374 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.195295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.195601 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.195558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.197314 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.197296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.197456 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.197440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.202241 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.202211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.202329 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.202319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn49b\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b\") pod \"e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") " pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.359274 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.359219    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:15.632426 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.632395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 04:58:15.634070 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:15.634044    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1979b7_fc13_41e1_b848_2ff667e4ea67.slice/crio-b4464ccfd1e90c6041f07ebcf430bf77bcb9ddfdd59a517c6f3ae5a19dc8414c WatchSource:0}: Error finding container b4464ccfd1e90c6041f07ebcf430bf77bcb9ddfdd59a517c6f3ae5a19dc8414c: Status 404 returned error can't find the container with id b4464ccfd1e90c6041f07ebcf430bf77bcb9ddfdd59a517c6f3ae5a19dc8414c
Sep 11 04:58:15.636270 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.636242    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:15.636362 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.636307    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:15.636362 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.636338    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:15.778285 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.778252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerStarted","Data":"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"}
Sep 11 04:58:15.778454 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.778302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerStarted","Data":"b4464ccfd1e90c6041f07ebcf430bf77bcb9ddfdd59a517c6f3ae5a19dc8414c"}
Sep 11 04:58:15.798029 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:15.797978    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podStartSLOduration=0.797962442 podStartE2EDuration="797.962442ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:15.795596624 +0000 UTC m=+1090.087071052" watchObservedRunningTime="2026-09-11 04:58:15.797962442 +0000 UTC m=+1090.089436878"
Sep 11 04:58:16.359623 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:16.359573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:16.364814 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:16.364789    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:16.783801 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:16.783774    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:16.784759 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:16.784738    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:23.540511 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.540479    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 04:58:23.543092 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.543073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.545764 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.545745    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"openshift-service-ca.crt\""
Sep 11 04:58:23.545867 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.545841    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"kube-root-ca.crt\""
Sep 11 04:58:23.545946 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.545929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"default-dockercfg-lxgst\""
Sep 11 04:58:23.546069 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.546005    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"auth-test-model-c1858a-kserve-self-signed-certs\""
Sep 11 04:58:23.555709 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.554047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 04:58:23.570629 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.570751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.570751 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.570867 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.570925 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.570925 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.570900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj68h\" (UniqueName: \"kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672702 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672702 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj68h\" (UniqueName: \"kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672966 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672880    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.672966 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.672891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.674709 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.674692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.675065 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.675045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.680767 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.680744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj68h\" (UniqueName: \"kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h\") pod \"auth-test-model-c1858a-kserve-85c89bf655-2764z\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:23.852872 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:23.852778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:24.120301 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:24.120274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 04:58:24.122247 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:24.122223    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode63ceda0_d391_48ca_86ce_e39d4fcab5b0.slice/crio-3b029e86f818ec721338fb4d8e16c470de97eff1f56dfec0ff0b78cf226e3303 WatchSource:0}: Error finding container 3b029e86f818ec721338fb4d8e16c470de97eff1f56dfec0ff0b78cf226e3303: Status 404 returned error can't find the container with id 3b029e86f818ec721338fb4d8e16c470de97eff1f56dfec0ff0b78cf226e3303
Sep 11 04:58:24.829986 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:24.829941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" event={"ID":"e63ceda0-d391-48ca-86ce-e39d4fcab5b0","Type":"ContainerStarted","Data":"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"}
Sep 11 04:58:24.830388 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:24.830005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" event={"ID":"e63ceda0-d391-48ca-86ce-e39d4fcab5b0","Type":"ContainerStarted","Data":"3b029e86f818ec721338fb4d8e16c470de97eff1f56dfec0ff0b78cf226e3303"}
Sep 11 04:58:24.849500 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:24.849452    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" podStartSLOduration=1.849436856 podStartE2EDuration="1.849436856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:24.849165343 +0000 UTC m=+1099.140639780" watchObservedRunningTime="2026-09-11 04:58:24.849436856 +0000 UTC m=+1099.140911296"
Sep 11 04:58:32.825729 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:32.825690    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:58:32.826172 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:32.825752    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:58:33.853431 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:33.853304    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:33.853431 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:33.853341    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:33.868602 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:33.866195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:33.905814 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:33.905791    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 04:58:34.901323 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.901293    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerID="020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1" exitCode=0
Sep 11 04:58:34.901718 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.901377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerDied","Data":"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"}
Sep 11 04:58:34.901915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.901897    2581 scope.go:117] "RemoveContainer" containerID="020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"
Sep 11 04:58:34.903006 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.902982    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:34.903062 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.903046    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:34.903101 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:34.903075    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:58:35.908968 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:35.908933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerStarted","Data":"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"}
Sep 11 04:58:36.359663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:36.359624    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:36.359913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:36.359872    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 11 04:58:36.359998 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:36.359942    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 11 04:58:37.360072 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:37.360040    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 11 04:58:37.360448 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:37.360088    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 11 04:58:38.360573 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:38.360537    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 11 04:58:38.360966 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:38.360712    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 11 04:58:39.360322 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:39.360273    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 11 04:58:39.360499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:39.360342    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 11 04:58:40.359810 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:40.359773    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 11 04:58:40.360190 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:40.359826    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 11 04:58:41.363894 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:41.363863    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:41.364319 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:41.364212    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:41.364841 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:41.364820    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 04:58:43.378535 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.378499    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7515-818b39"]
Sep 11 04:58:43.380989 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.380974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:43.383025 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.383001    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mgwvk\""
Sep 11 04:58:43.385890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.385863    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-818b39"]
Sep 11 04:58:43.444524 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.444494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfxhm\" (UniqueName: \"kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm\") pod \"test-curl-7515-818b39\" (UID: \"755b9286-f582-4ff0-83ef-e062f46fb2b5\") " pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:43.545929 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.545905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfxhm\" (UniqueName: \"kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm\") pod \"test-curl-7515-818b39\" (UID: \"755b9286-f582-4ff0-83ef-e062f46fb2b5\") " pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:43.553462 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.553444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfxhm\" (UniqueName: \"kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm\") pod \"test-curl-7515-818b39\" (UID: \"755b9286-f582-4ff0-83ef-e062f46fb2b5\") " pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:43.692375 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.692322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:43.963701 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.963675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-818b39"]
Sep 11 04:58:43.964825 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:43.964793    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod755b9286_f582_4ff0_83ef_e062f46fb2b5.slice/crio-d9833c5d1dea01b599a069900953644e9e9d4d1b1a504c8a739d04e9ea6982ee WatchSource:0}: Error finding container d9833c5d1dea01b599a069900953644e9e9d4d1b1a504c8a739d04e9ea6982ee: Status 404 returned error can't find the container with id d9833c5d1dea01b599a069900953644e9e9d4d1b1a504c8a739d04e9ea6982ee
Sep 11 04:58:43.972720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:43.972697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-818b39" event={"ID":"755b9286-f582-4ff0-83ef-e062f46fb2b5","Type":"ContainerStarted","Data":"d9833c5d1dea01b599a069900953644e9e9d4d1b1a504c8a739d04e9ea6982ee"}
Sep 11 04:58:44.979695 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:44.979614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-818b39" event={"ID":"755b9286-f582-4ff0-83ef-e062f46fb2b5","Type":"ContainerStarted","Data":"a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c"}
Sep 11 04:58:44.994825 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:44.994779    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7515-818b39" podStartSLOduration=1.994762767 podStartE2EDuration="1.994762767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:44.992348499 +0000 UTC m=+1119.283822936" watchObservedRunningTime="2026-09-11 04:58:44.994762767 +0000 UTC m=+1119.286237267"
Sep 11 04:58:45.570530 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.570481    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7515-818b39"]
Sep 11 04:58:45.574981 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.574955    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7515-818b39"]
Sep 11 04:58:45.685559 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.685491    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7515-8a36f4"]
Sep 11 04:58:45.685897 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.685883    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" containerName="test-curl-7515-818b39"
Sep 11 04:58:45.685947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.685902    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" containerName="test-curl-7515-818b39"
Sep 11 04:58:45.686030 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.685960    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" containerName="test-curl-7515-818b39"
Sep 11 04:58:45.688135 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.688120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:45.692250 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.692229    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-8a36f4"]
Sep 11 04:58:45.767534 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.767506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhlw\" (UniqueName: \"kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw\") pod \"test-curl-7515-8a36f4\" (UID: \"51216a1c-3442-431e-9cb1-cc99d79eeafd\") " pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:45.868082 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.868048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhlw\" (UniqueName: \"kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw\") pod \"test-curl-7515-8a36f4\" (UID: \"51216a1c-3442-431e-9cb1-cc99d79eeafd\") " pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:45.875775 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.875751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhlw\" (UniqueName: \"kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw\") pod \"test-curl-7515-8a36f4\" (UID: \"51216a1c-3442-431e-9cb1-cc99d79eeafd\") " pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:45.985435 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:45.985412    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7515-818b39" container="test-curl-7515-818b39"
Sep 11 04:58:45.998636 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:45.998619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:46.265735 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:46.265711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-8a36f4"]
Sep 11 04:58:46.267229 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:46.267189    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51216a1c_3442_431e_9cb1_cc99d79eeafd.slice/crio-07564cc590ba31be859dec620a55f2b89cd3cad3f6c676a81d81a3dbed332846 WatchSource:0}: Error finding container 07564cc590ba31be859dec620a55f2b89cd3cad3f6c676a81d81a3dbed332846: Status 404 returned error can't find the container with id 07564cc590ba31be859dec620a55f2b89cd3cad3f6c676a81d81a3dbed332846
Sep 11 04:58:46.992888 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:46.992844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-8a36f4" event={"ID":"51216a1c-3442-431e-9cb1-cc99d79eeafd","Type":"ContainerStarted","Data":"9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb"}
Sep 11 04:58:46.993310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:46.992901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-8a36f4" event={"ID":"51216a1c-3442-431e-9cb1-cc99d79eeafd","Type":"ContainerStarted","Data":"07564cc590ba31be859dec620a55f2b89cd3cad3f6c676a81d81a3dbed332846"}
Sep 11 04:58:46.993310 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:46.992989    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7515-818b39" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" containerName="test-curl-7515-818b39" containerID="cri-o://a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c" gracePeriod=2
Sep 11 04:58:47.005826 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.005781    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7515-8a36f4" podStartSLOduration=2.005765799 podStartE2EDuration="2.005765799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:47.005047321 +0000 UTC m=+1121.296521760" watchObservedRunningTime="2026-09-11 04:58:47.005765799 +0000 UTC m=+1121.297240237"
Sep 11 04:58:47.574661 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.574628    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7515-8a36f4"]
Sep 11 04:58:47.577290 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.577267    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7515-8a36f4"]
Sep 11 04:58:47.688797 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.688736    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7515-216227"]
Sep 11 04:58:47.689079 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.689065    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" containerName="test-curl-7515-8a36f4"
Sep 11 04:58:47.689129 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.689081    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" containerName="test-curl-7515-8a36f4"
Sep 11 04:58:47.689166 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.689142    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" containerName="test-curl-7515-8a36f4"
Sep 11 04:58:47.690979 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.690964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:47.697284 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.697263    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-216227"]
Sep 11 04:58:47.784662 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.784639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqvkl\" (UniqueName: \"kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl\") pod \"test-curl-7515-216227\" (UID: \"f7289c68-9fea-4e30-a298-91011a8833f5\") " pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:47.885380 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.885354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqvkl\" (UniqueName: \"kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl\") pod \"test-curl-7515-216227\" (UID: \"f7289c68-9fea-4e30-a298-91011a8833f5\") " pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:47.892908 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:47.892884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqvkl\" (UniqueName: \"kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl\") pod \"test-curl-7515-216227\" (UID: \"f7289c68-9fea-4e30-a298-91011a8833f5\") " pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:47.998783 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:47.998763    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7515-8a36f4" container="test-curl-7515-8a36f4"
Sep 11 04:58:48.001516 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:48.001496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:48.269663 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:48.269641    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-216227"]
Sep 11 04:58:48.271821 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:48.271795    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7289c68_9fea_4e30_a298_91011a8833f5.slice/crio-6cc144c9b42f062de4e28da827b1ea428afb0f5c588828068cef9e2f93cb37ef WatchSource:0}: Error finding container 6cc144c9b42f062de4e28da827b1ea428afb0f5c588828068cef9e2f93cb37ef: Status 404 returned error can't find the container with id 6cc144c9b42f062de4e28da827b1ea428afb0f5c588828068cef9e2f93cb37ef
Sep 11 04:58:49.006174 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.006143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-216227" event={"ID":"f7289c68-9fea-4e30-a298-91011a8833f5","Type":"ContainerStarted","Data":"57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6"}
Sep 11 04:58:49.006493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.006193    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-216227" event={"ID":"f7289c68-9fea-4e30-a298-91011a8833f5","Type":"ContainerStarted","Data":"6cc144c9b42f062de4e28da827b1ea428afb0f5c588828068cef9e2f93cb37ef"}
Sep 11 04:58:49.006493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.006216    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7515-8a36f4" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" containerName="test-curl-7515-8a36f4" containerID="cri-o://9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb" gracePeriod=2
Sep 11 04:58:49.021024 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.020982    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7515-216227" podStartSLOduration=2.020968879 podStartE2EDuration="2.020968879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:49.019966644 +0000 UTC m=+1123.311441081" watchObservedRunningTime="2026-09-11 04:58:49.020968879 +0000 UTC m=+1123.312443315"
Sep 11 04:58:49.176864 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.176841    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:49.297829 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.297761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfxhm\" (UniqueName: \"kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm\") pod \"755b9286-f582-4ff0-83ef-e062f46fb2b5\" (UID: \"755b9286-f582-4ff0-83ef-e062f46fb2b5\") "
Sep 11 04:58:49.299723 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.299701    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm" (OuterVolumeSpecName: "kube-api-access-dfxhm") pod "755b9286-f582-4ff0-83ef-e062f46fb2b5" (UID: "755b9286-f582-4ff0-83ef-e062f46fb2b5"). InnerVolumeSpecName "kube-api-access-dfxhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:58:49.398531 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.398511    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfxhm\" (UniqueName: \"kubernetes.io/projected/755b9286-f582-4ff0-83ef-e062f46fb2b5-kube-api-access-dfxhm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:58:49.599230 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.599146    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7515-216227"]
Sep 11 04:58:49.602672 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.602647    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7515-216227"]
Sep 11 04:58:49.725613 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.725571    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7515-5ec4a4"]
Sep 11 04:58:49.725943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.725932    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" containerName="test-curl-7515-216227"
Sep 11 04:58:49.725988 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.725945    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" containerName="test-curl-7515-216227"
Sep 11 04:58:49.726020 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.726003    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" containerName="test-curl-7515-216227"
Sep 11 04:58:49.727818 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.727804    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:49.733930 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.733902    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-5ec4a4"]
Sep 11 04:58:49.802119 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.802090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrq5\" (UniqueName: \"kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5\") pod \"test-curl-7515-5ec4a4\" (UID: \"0efd4dfc-877d-4877-b4b2-5eabccf840c2\") " pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:49.903375 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.903319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrq5\" (UniqueName: \"kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5\") pod \"test-curl-7515-5ec4a4\" (UID: \"0efd4dfc-877d-4877-b4b2-5eabccf840c2\") " pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:49.912199 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:49.912166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrq5\" (UniqueName: \"kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5\") pod \"test-curl-7515-5ec4a4\" (UID: \"0efd4dfc-877d-4877-b4b2-5eabccf840c2\") " pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:50.013873 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.013851    2581 generic.go:356] "Generic (PLEG): container finished" podID="755b9286-f582-4ff0-83ef-e062f46fb2b5" containerID="a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c" exitCode=137
Sep 11 04:58:50.014204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.013890    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:50.014204 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.013967    2581 scope.go:117] "RemoveContainer" containerID="a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c"
Sep 11 04:58:50.014204 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:50.014144    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7515-216227" container="test-curl-7515-216227"
Sep 11 04:58:50.022894 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.022874    2581 scope.go:117] "RemoveContainer" containerID="a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c"
Sep 11 04:58:50.023151 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:50.023135    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c\": container with ID starting with a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c not found: ID does not exist" containerID="a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c"
Sep 11 04:58:50.023201 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.023157    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c"} err="failed to get container status \"a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c\": rpc error: code = NotFound desc = could not find container \"a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c\": container with ID starting with a85d188da159ce934fc26ee7cdb698daeebb23a7c4f18315f859354430f9743c not found: ID does not exist"
Sep 11 04:58:50.024881 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:50.024861    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-818b39\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" pod="opendatahub/test-curl-7515-818b39"
Sep 11 04:58:50.037983 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.037961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:50.192698 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.192635    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755b9286-f582-4ff0-83ef-e062f46fb2b5" path="/var/lib/kubelet/pods/755b9286-f582-4ff0-83ef-e062f46fb2b5/volumes"
Sep 11 04:58:50.321315 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:50.321292    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7515-5ec4a4"]
Sep 11 04:58:50.323410 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:58:50.323384    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0efd4dfc_877d_4877_b4b2_5eabccf840c2.slice/crio-29287a19d8242fbf86ff81ed969b11d651ce2584e118e76dedcba91e58467ed3 WatchSource:0}: Error finding container 29287a19d8242fbf86ff81ed969b11d651ce2584e118e76dedcba91e58467ed3: Status 404 returned error can't find the container with id 29287a19d8242fbf86ff81ed969b11d651ce2584e118e76dedcba91e58467ed3
Sep 11 04:58:51.022947 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.022920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-5ec4a4" event={"ID":"0efd4dfc-877d-4877-b4b2-5eabccf840c2","Type":"ContainerStarted","Data":"4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592"}
Sep 11 04:58:51.023297 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.022965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7515-5ec4a4" event={"ID":"0efd4dfc-877d-4877-b4b2-5eabccf840c2","Type":"ContainerStarted","Data":"29287a19d8242fbf86ff81ed969b11d651ce2584e118e76dedcba91e58467ed3"}
Sep 11 04:58:51.023943 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.023921    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7515-216227" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" containerName="test-curl-7515-216227" containerID="cri-o://57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6" gracePeriod=2
Sep 11 04:58:51.043694 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.043658    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7515-5ec4a4" podStartSLOduration=2.043645251 podStartE2EDuration="2.043645251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:51.043559564 +0000 UTC m=+1125.335034001" watchObservedRunningTime="2026-09-11 04:58:51.043645251 +0000 UTC m=+1125.335119687"
Sep 11 04:58:51.169924 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.169904    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:51.315619 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.315528    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxhlw\" (UniqueName: \"kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw\") pod \"51216a1c-3442-431e-9cb1-cc99d79eeafd\" (UID: \"51216a1c-3442-431e-9cb1-cc99d79eeafd\") "
Sep 11 04:58:51.317551 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.317526    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw" (OuterVolumeSpecName: "kube-api-access-nxhlw") pod "51216a1c-3442-431e-9cb1-cc99d79eeafd" (UID: "51216a1c-3442-431e-9cb1-cc99d79eeafd"). InnerVolumeSpecName "kube-api-access-nxhlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:58:51.416699 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.416677    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxhlw\" (UniqueName: \"kubernetes.io/projected/51216a1c-3442-431e-9cb1-cc99d79eeafd-kube-api-access-nxhlw\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:58:51.635720 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.635627    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7515-5ec4a4"]
Sep 11 04:58:51.638015 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:51.637992    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7515-5ec4a4"]
Sep 11 04:58:52.030394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.030353    2581 generic.go:356] "Generic (PLEG): container finished" podID="51216a1c-3442-431e-9cb1-cc99d79eeafd" containerID="9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb" exitCode=137
Sep 11 04:58:52.030394 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.030397    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:52.030915 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.030458    2581 scope.go:117] "RemoveContainer" containerID="9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb"
Sep 11 04:58:52.030915 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:52.030780    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7515-5ec4a4" container="test-curl-7515-5ec4a4"
Sep 11 04:58:52.042763 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.042746    2581 scope.go:117] "RemoveContainer" containerID="9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb"
Sep 11 04:58:52.043081 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:52.043054    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb\": container with ID starting with 9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb not found: ID does not exist" containerID="9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb"
Sep 11 04:58:52.043180 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.043079    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb"} err="failed to get container status \"9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb\": rpc error: code = NotFound desc = could not find container \"9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb\": container with ID starting with 9d0226905c3f53f79752dace1d01b7a8450477cbebcbd2c397b19c2ccc14d1bb not found: ID does not exist"
Sep 11 04:58:52.043356 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:52.043339    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-8a36f4\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" pod="opendatahub/test-curl-7515-8a36f4"
Sep 11 04:58:52.192562 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:52.192531    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51216a1c-3442-431e-9cb1-cc99d79eeafd" path="/var/lib/kubelet/pods/51216a1c-3442-431e-9cb1-cc99d79eeafd/volumes"
Sep 11 04:58:53.037499 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:53.037471    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7515-5ec4a4" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" containerName="test-curl-7515-5ec4a4" containerID="cri-o://4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592" gracePeriod=2
Sep 11 04:58:53.194121 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:53.194101    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:53.334427 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:53.334366    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqvkl\" (UniqueName: \"kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl\") pod \"f7289c68-9fea-4e30-a298-91011a8833f5\" (UID: \"f7289c68-9fea-4e30-a298-91011a8833f5\") "
Sep 11 04:58:53.336493 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:53.336472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl" (OuterVolumeSpecName: "kube-api-access-fqvkl") pod "f7289c68-9fea-4e30-a298-91011a8833f5" (UID: "f7289c68-9fea-4e30-a298-91011a8833f5"). InnerVolumeSpecName "kube-api-access-fqvkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:58:53.435815 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:53.435787    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqvkl\" (UniqueName: \"kubernetes.io/projected/f7289c68-9fea-4e30-a298-91011a8833f5-kube-api-access-fqvkl\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:58:54.043913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.043878    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7289c68-9fea-4e30-a298-91011a8833f5" containerID="57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6" exitCode=137
Sep 11 04:58:54.043913 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.043916    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:54.044396 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.043968    2581 scope.go:117] "RemoveContainer" containerID="57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6"
Sep 11 04:58:54.053928 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.053910    2581 scope.go:117] "RemoveContainer" containerID="57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6"
Sep 11 04:58:54.054133 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:54.054107    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-216227\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" pod="opendatahub/test-curl-7515-216227"
Sep 11 04:58:54.054199 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:54.054182    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6\": container with ID starting with 57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6 not found: ID does not exist" containerID="57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6"
Sep 11 04:58:54.054240 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.054206    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6"} err="failed to get container status \"57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6\": rpc error: code = NotFound desc = could not find container \"57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6\": container with ID starting with 57a0a49e0b2c9a78c61bfe76a831d620075a197449e621d26bdbb84fca877ee6 not found: ID does not exist"
Sep 11 04:58:54.191665 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:54.191640    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7289c68-9fea-4e30-a298-91011a8833f5" path="/var/lib/kubelet/pods/f7289c68-9fea-4e30-a298-91011a8833f5/volumes"
Sep 11 04:58:55.173692 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:55.173670    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:55.251954 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:55.251917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxrq5\" (UniqueName: \"kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5\") pod \"0efd4dfc-877d-4877-b4b2-5eabccf840c2\" (UID: \"0efd4dfc-877d-4877-b4b2-5eabccf840c2\") "
Sep 11 04:58:55.253900 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:55.253876    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5" (OuterVolumeSpecName: "kube-api-access-xxrq5") pod "0efd4dfc-877d-4877-b4b2-5eabccf840c2" (UID: "0efd4dfc-877d-4877-b4b2-5eabccf840c2"). InnerVolumeSpecName "kube-api-access-xxrq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:58:55.353404 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:55.353340    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrq5\" (UniqueName: \"kubernetes.io/projected/0efd4dfc-877d-4877-b4b2-5eabccf840c2-kube-api-access-xxrq5\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 04:58:56.059059 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.059022    2581 generic.go:356] "Generic (PLEG): container finished" podID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" containerID="4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592" exitCode=137
Sep 11 04:58:56.059238 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.059075    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:56.059238 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.059124    2581 scope.go:117] "RemoveContainer" containerID="4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592"
Sep 11 04:58:56.067929 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.067912    2581 scope.go:117] "RemoveContainer" containerID="4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592"
Sep 11 04:58:56.068184 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:56.068166    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592\": container with ID starting with 4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592 not found: ID does not exist" containerID="4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592"
Sep 11 04:58:56.068232 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.068191    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592"} err="failed to get container status \"4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592\": rpc error: code = NotFound desc = could not find container \"4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592\": container with ID starting with 4c9dd7268352ccdfd709e3f9849f4424c3e2ecd3cc83734828c674c876277592 not found: ID does not exist"
Sep 11 04:58:56.068541 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:56.068522    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-5ec4a4\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:56.192073 ip-10-0-139-137 kubenswrapper[2581]: E0911 04:58:56.192048    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7515-5ec4a4\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" pod="opendatahub/test-curl-7515-5ec4a4"
Sep 11 04:58:56.192862 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:58:56.192844    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" path="/var/lib/kubelet/pods/0efd4dfc-877d-4877-b4b2-5eabccf840c2/volumes"
Sep 11 04:59:01.860597 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.860545    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 04:59:01.860974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.860880    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" containerName="test-curl-7515-5ec4a4"
Sep 11 04:59:01.860974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.860892    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" containerName="test-curl-7515-5ec4a4"
Sep 11 04:59:01.860974 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.860960    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0efd4dfc-877d-4877-b4b2-5eabccf840c2" containerName="test-curl-7515-5ec4a4"
Sep 11 04:59:01.863714 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.863698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:01.866545 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.866523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"auth-test-model-16b799-kserve-self-signed-certs\""
Sep 11 04:59:01.866667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.866523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"default-dockercfg-vwxtm\""
Sep 11 04:59:01.866667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.866527    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"kube-root-ca.crt\""
Sep 11 04:59:01.866667 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.866527    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"openshift-service-ca.crt\""
Sep 11 04:59:01.873757 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:01.873740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 04:59:02.010073 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.010229 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.010229 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpv2\" (UniqueName: \"kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.010229 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.010369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.010369 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.010285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110744 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110744 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpv2\" (UniqueName: \"kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.110960 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.110901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.111165 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.111079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.111165 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.111156    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.111165 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.111157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.113186 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.113167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.113568 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.113549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.118331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.118306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpv2\" (UniqueName: \"kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2\") pod \"auth-test-model-16b799-kserve-f585f647f-8t6qs\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") " pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.173890 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.173854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:02.451805 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:02.451782    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 04:59:02.453491 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:59:02.453466    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0902714_6744_42cf_829c_2215ff08e8ba.slice/crio-91654945abbddd1479514040f8f1b13bb61af0a2ec013f38b33ada20496982cf WatchSource:0}: Error finding container 91654945abbddd1479514040f8f1b13bb61af0a2ec013f38b33ada20496982cf: Status 404 returned error can't find the container with id 91654945abbddd1479514040f8f1b13bb61af0a2ec013f38b33ada20496982cf
Sep 11 04:59:03.100785 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:03.100750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" event={"ID":"d0902714-6744-42cf-829c-2215ff08e8ba","Type":"ContainerStarted","Data":"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"}
Sep 11 04:59:03.101208 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:03.100801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" event={"ID":"d0902714-6744-42cf-829c-2215ff08e8ba","Type":"ContainerStarted","Data":"91654945abbddd1479514040f8f1b13bb61af0a2ec013f38b33ada20496982cf"}
Sep 11 04:59:03.119942 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:03.119888    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" podStartSLOduration=2.119867197 podStartE2EDuration="2.119867197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:59:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:59:03.117626706 +0000 UTC m=+1137.409101137" watchObservedRunningTime="2026-09-11 04:59:03.119867197 +0000 UTC m=+1137.411341634"
Sep 11 04:59:06.284917 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:06.284888    2581 scope.go:117] "RemoveContainer" containerID="51ca7033b01efaf5948beb19975a4d02bc3234d59159f2c8f69c299eca4717c9"
Sep 11 04:59:12.174835 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:12.174796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:12.175277 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:12.174854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:12.192922 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:12.192888    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:13.170944 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:13.170918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 04:59:46.754109 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.754074    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 04:59:46.757528 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.757511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.759992 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.759972    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-89e681d0\"/\"openshift-service-ca.crt\""
Sep 11 04:59:46.760455 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.760437    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-89e681d0\"/\"e2e-lifecycle-model-89e681d0-kserve-self-signed-certs\""
Sep 11 04:59:46.760455 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.760448    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-89e681d0\"/\"kube-root-ca.crt\""
Sep 11 04:59:46.760596 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.760534    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-89e681d0\"/\"default-dockercfg-9gx27\""
Sep 11 04:59:46.766544 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.766522    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 04:59:46.857843 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.857805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.858026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.857874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.858026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.857953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.858026 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.857987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njwgn\" (UniqueName: \"kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.858138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.858035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.858138 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.858055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.958877 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.958828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.958877 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.958886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njwgn\" (UniqueName: \"kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.958926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.958948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.958986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959111 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.959080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959335 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.959315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959375 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.959355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.959422 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.959386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.961549 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.961517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.961687 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.961657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:46.967057 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:46.967036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njwgn\" (UniqueName: \"kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn\") pod \"e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") " pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:47.068331 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:47.068240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:47.341705 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:47.341684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 04:59:47.343546 ip-10-0-139-137 kubenswrapper[2581]: W0911 04:59:47.343518    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod890f3d35_7da0_4d65_832b_79617f1bbeb5.slice/crio-10f308a3ee28e504ed15154064750f70342ccdd4589545e637fafdbdfd13bfbe WatchSource:0}: Error finding container 10f308a3ee28e504ed15154064750f70342ccdd4589545e637fafdbdfd13bfbe: Status 404 returned error can't find the container with id 10f308a3ee28e504ed15154064750f70342ccdd4589545e637fafdbdfd13bfbe
Sep 11 04:59:48.347814 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:48.347778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" event={"ID":"890f3d35-7da0-4d65-832b-79617f1bbeb5","Type":"ContainerStarted","Data":"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"}
Sep 11 04:59:48.347814 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:48.347818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" event={"ID":"890f3d35-7da0-4d65-832b-79617f1bbeb5","Type":"ContainerStarted","Data":"10f308a3ee28e504ed15154064750f70342ccdd4589545e637fafdbdfd13bfbe"}
Sep 11 04:59:48.371959 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:48.371903    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" podStartSLOduration=2.371887811 podStartE2EDuration="2.371887811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:59:48.371427653 +0000 UTC m=+1182.662902091" watchObservedRunningTime="2026-09-11 04:59:48.371887811 +0000 UTC m=+1182.663362249"
Sep 11 04:59:57.069183 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:57.069150    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:57.069669 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:57.069205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:57.081383 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:57.081360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 04:59:57.406694 ip-10-0-139-137 kubenswrapper[2581]: I0911 04:59:57.406621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 05:00:00.148271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.148237    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"]
Sep 11 05:00:00.151761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.151732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"
Sep 11 05:00:00.153313 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.153269    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"]
Sep 11 05:00:00.154549 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.154531    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:00:00.163302 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.163239    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"]
Sep 11 05:00:00.165784 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.164532    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"
Sep 11 05:00:00.167154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.167003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"
Sep 11 05:00:00.168298 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.168266    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"]
Sep 11 05:00:00.170386 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.170364    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"
Sep 11 05:00:00.176468 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.176038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"]
Sep 11 05:00:00.176468 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.176074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"]
Sep 11 05:00:00.176468 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.176152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"
Sep 11 05:00:00.178516 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.178486    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"]
Sep 11 05:00:00.184524 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.184404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"
Sep 11 05:00:00.198334 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.198307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"]
Sep 11 05:00:00.222119 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.222095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"
Sep 11 05:00:00.229591 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.229549    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"
Sep 11 05:00:00.462101 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.461986    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"]
Sep 11 05:00:00.465395 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:00:00.465369    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948d3c1b_4f1d_4c84_b88f_848ff466a88a.slice/crio-38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e WatchSource:0}: Error finding container 38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e: Status 404 returned error can't find the container with id 38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e
Sep 11 05:00:00.481521 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.481488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"]
Sep 11 05:00:00.481957 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:00:00.481935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39acb8f8_2e2c_40a4_b566_8c0aa89596ff.slice/crio-834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482 WatchSource:0}: Error finding container 834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482: Status 404 returned error can't find the container with id 834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482
Sep 11 05:00:00.530104 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.530078    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"]
Sep 11 05:00:00.531810 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:00:00.531779    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7909a84a_4552_4016_8a73_ea0299df6383.slice/crio-49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab WatchSource:0}: Error finding container 49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab: Status 404 returned error can't find the container with id 49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab
Sep 11 05:00:00.532512 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:00.532484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"]
Sep 11 05:00:00.534152 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:00:00.534128    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b5fa7c_2438_4531_b8bc_fff54661eafb.slice/crio-513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003 WatchSource:0}: Error finding container 513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003: Status 404 returned error can't find the container with id 513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003
Sep 11 05:00:01.422076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.422045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj" event={"ID":"39acb8f8-2e2c-40a4-b566-8c0aa89596ff","Type":"ContainerStarted","Data":"9bf59e3955b4e8add1c81f9123b0db176aeb3b97ff32f42ac34d84d2167e6a6a"}
Sep 11 05:00:01.422496 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.422107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj" event={"ID":"39acb8f8-2e2c-40a4-b566-8c0aa89596ff","Type":"ContainerStarted","Data":"834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482"}
Sep 11 05:00:01.423843 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.423810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx" event={"ID":"7909a84a-4552-4016-8a73-ea0299df6383","Type":"ContainerStarted","Data":"b262441928f99e08535f2bf5ab480d6a631b198173906e02b50dc087cb3b58c5"}
Sep 11 05:00:01.424012 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.423862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx" event={"ID":"7909a84a-4552-4016-8a73-ea0299df6383","Type":"ContainerStarted","Data":"49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab"}
Sep 11 05:00:01.425649 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.425617    2581 generic.go:356] "Generic (PLEG): container finished" podID="948d3c1b-4f1d-4c84-b88f-848ff466a88a" containerID="8a1a166b8f8ad40c270191011984f7887b7c4805ef969cc11c997bfaac323d11" exitCode=0
Sep 11 05:00:01.425762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.425702    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm" event={"ID":"948d3c1b-4f1d-4c84-b88f-848ff466a88a","Type":"ContainerDied","Data":"8a1a166b8f8ad40c270191011984f7887b7c4805ef969cc11c997bfaac323d11"}
Sep 11 05:00:01.425762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.425741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm" event={"ID":"948d3c1b-4f1d-4c84-b88f-848ff466a88a","Type":"ContainerStarted","Data":"38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e"}
Sep 11 05:00:01.427485 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.427453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874" event={"ID":"39b5fa7c-2438-4531-b8bc-fff54661eafb","Type":"ContainerStarted","Data":"8c07a3e73e2c82f12efeb95a05ea8ca453d5c0f3f83c1e0eac155e95352000e3"}
Sep 11 05:00:01.427595 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.427500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874" event={"ID":"39b5fa7c-2438-4531-b8bc-fff54661eafb","Type":"ContainerStarted","Data":"513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003"}
Sep 11 05:00:01.440999 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.440961    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj" podStartSLOduration=1.4409480239999999 podStartE2EDuration="1.440948024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:00:01.437978131 +0000 UTC m=+1195.729452567" watchObservedRunningTime="2026-09-11 05:00:01.440948024 +0000 UTC m=+1195.732422461"
Sep 11 05:00:01.455202 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.455158    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx" podStartSLOduration=1.455147272 podStartE2EDuration="1.455147272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:00:01.454228745 +0000 UTC m=+1195.745703181" watchObservedRunningTime="2026-09-11 05:00:01.455147272 +0000 UTC m=+1195.746621714"
Sep 11 05:00:01.482806 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:01.482773    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874" podStartSLOduration=1.4827620320000001 podStartE2EDuration="1.482762032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:00:01.481089685 +0000 UTC m=+1195.772564122" watchObservedRunningTime="2026-09-11 05:00:01.482762032 +0000 UTC m=+1195.774236518"
Sep 11 05:00:02.434302 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.434262    2581 generic.go:356] "Generic (PLEG): container finished" podID="7909a84a-4552-4016-8a73-ea0299df6383" containerID="b262441928f99e08535f2bf5ab480d6a631b198173906e02b50dc087cb3b58c5" exitCode=0
Sep 11 05:00:02.434753 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.434349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx" event={"ID":"7909a84a-4552-4016-8a73-ea0299df6383","Type":"ContainerDied","Data":"b262441928f99e08535f2bf5ab480d6a631b198173906e02b50dc087cb3b58c5"}
Sep 11 05:00:02.435882 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.435861    2581 generic.go:356] "Generic (PLEG): container finished" podID="39b5fa7c-2438-4531-b8bc-fff54661eafb" containerID="8c07a3e73e2c82f12efeb95a05ea8ca453d5c0f3f83c1e0eac155e95352000e3" exitCode=0
Sep 11 05:00:02.435996 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.435923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874" event={"ID":"39b5fa7c-2438-4531-b8bc-fff54661eafb","Type":"ContainerDied","Data":"8c07a3e73e2c82f12efeb95a05ea8ca453d5c0f3f83c1e0eac155e95352000e3"}
Sep 11 05:00:02.437432 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.437412    2581 generic.go:356] "Generic (PLEG): container finished" podID="39acb8f8-2e2c-40a4-b566-8c0aa89596ff" containerID="9bf59e3955b4e8add1c81f9123b0db176aeb3b97ff32f42ac34d84d2167e6a6a" exitCode=0
Sep 11 05:00:02.437543 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.437477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj" event={"ID":"39acb8f8-2e2c-40a4-b566-8c0aa89596ff","Type":"ContainerDied","Data":"9bf59e3955b4e8add1c81f9123b0db176aeb3b97ff32f42ac34d84d2167e6a6a"}
Sep 11 05:00:02.588051 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:02.588027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"
Sep 11 05:00:03.444672 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.444646    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"
Sep 11 05:00:03.445127 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.444693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm" event={"ID":"948d3c1b-4f1d-4c84-b88f-848ff466a88a","Type":"ContainerDied","Data":"38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e"}
Sep 11 05:00:03.445127 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.444729    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38eead4b7d023a0ab6fa03639c276824829a2914c1bde4bfbb23bc32fb0e446e"
Sep 11 05:00:03.592041 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.592016    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"
Sep 11 05:00:03.637653 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.637629    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"
Sep 11 05:00:03.640569 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:03.640553    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"
Sep 11 05:00:04.451726 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.451701    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"
Sep 11 05:00:04.451726 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.451723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj" event={"ID":"39acb8f8-2e2c-40a4-b566-8c0aa89596ff","Type":"ContainerDied","Data":"834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482"}
Sep 11 05:00:04.452188 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.451777    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="834d8c9b6526a5967fc0821ea9852ec99b2fbecfd4fde0084134224546827482"
Sep 11 05:00:04.453181 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.453121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"
Sep 11 05:00:04.453181 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.453159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx" event={"ID":"7909a84a-4552-4016-8a73-ea0299df6383","Type":"ContainerDied","Data":"49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab"}
Sep 11 05:00:04.453364 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.453206    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49ab780ea43ee097b8e6f3587d2482450e6c4da3749416ea406a65f94c21ebab"
Sep 11 05:00:04.454622 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.454602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874" event={"ID":"39b5fa7c-2438-4531-b8bc-fff54661eafb","Type":"ContainerDied","Data":"513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003"}
Sep 11 05:00:04.454622 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.454617    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818380-q2874"
Sep 11 05:00:04.454756 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:04.454633    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="513a52e63bf90b52935d3d2c554eee64c3ca62edd262c36372d929d8cee9f003"
Sep 11 05:00:06.218215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:06.218177    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:00:06.220414 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:06.220378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:00:06.320082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:06.320053    2581 scope.go:117] "RemoveContainer" containerID="80e53b4d4a787c4e830d60dc1761f794f6cb93b9f6683c463ce41195f85e8d83"
Sep 11 05:00:06.328805 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:06.328787    2581 scope.go:117] "RemoveContainer" containerID="f92e880b82fef43fbcda6ed7e9f2056c1428caf4a12179ce6c0f7d2e1275a1ba"
Sep 11 05:00:14.991304 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991269    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"]
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991621    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b5fa7c-2438-4531-b8bc-fff54661eafb" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991633    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b5fa7c-2438-4531-b8bc-fff54661eafb" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991642    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7909a84a-4552-4016-8a73-ea0299df6383" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991647    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7909a84a-4552-4016-8a73-ea0299df6383" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991655    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39acb8f8-2e2c-40a4-b566-8c0aa89596ff" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991660    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="39acb8f8-2e2c-40a4-b566-8c0aa89596ff" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991671    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948d3c1b-4f1d-4c84-b88f-848ff466a88a" containerName="cleanup"
Sep 11 05:00:14.991728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991676    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="948d3c1b-4f1d-4c84-b88f-848ff466a88a" containerName="cleanup"
Sep 11 05:00:14.992002 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991734    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="948d3c1b-4f1d-4c84-b88f-848ff466a88a" containerName="cleanup"
Sep 11 05:00:14.992002 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991743    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b5fa7c-2438-4531-b8bc-fff54661eafb" containerName="cleanup"
Sep 11 05:00:14.992002 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991749    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="39acb8f8-2e2c-40a4-b566-8c0aa89596ff" containerName="cleanup"
Sep 11 05:00:14.992002 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.991755    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7909a84a-4552-4016-8a73-ea0299df6383" containerName="cleanup"
Sep 11 05:00:14.996249 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.996230    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:14.998678 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.998656    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:00:14.998925 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:14.998908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:00:15.005430 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.005404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"]
Sep 11 05:00:15.097596 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.097549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-89e681d0-99t7t\" (UID: \"9ebbe160-29a5-425e-83a2-ebb451f2b8aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:15.198382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.198349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-89e681d0-99t7t\" (UID: \"9ebbe160-29a5-425e-83a2-ebb451f2b8aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:15.198972 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.198953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-89e681d0-99t7t\" (UID: \"9ebbe160-29a5-425e-83a2-ebb451f2b8aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:15.307402 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.307324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:15.584077 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:15.584048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"]
Sep 11 05:00:15.584687 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:00:15.584658    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebbe160_29a5_425e_83a2_ebb451f2b8aa.slice/crio-4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb WatchSource:0}: Error finding container 4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb: Status 404 returned error can't find the container with id 4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb
Sep 11 05:00:16.522533 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:16.522506    2581 generic.go:356] "Generic (PLEG): container finished" podID="9ebbe160-29a5-425e-83a2-ebb451f2b8aa" containerID="5ea4cf930d1e4d6bda0d122ff28b9ba3348ea928341bfa29add2248442d9cb4a" exitCode=0
Sep 11 05:00:16.522930 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:16.522610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t" event={"ID":"9ebbe160-29a5-425e-83a2-ebb451f2b8aa","Type":"ContainerDied","Data":"5ea4cf930d1e4d6bda0d122ff28b9ba3348ea928341bfa29add2248442d9cb4a"}
Sep 11 05:00:16.522930 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:16.522664    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t" event={"ID":"9ebbe160-29a5-425e-83a2-ebb451f2b8aa","Type":"ContainerStarted","Data":"4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb"}
Sep 11 05:00:17.657623 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:17.657599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:17.824223 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:17.824146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca\") pod \"9ebbe160-29a5-425e-83a2-ebb451f2b8aa\" (UID: \"9ebbe160-29a5-425e-83a2-ebb451f2b8aa\") "
Sep 11 05:00:17.824485 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:17.824463    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ebbe160-29a5-425e-83a2-ebb451f2b8aa" (UID: "9ebbe160-29a5-425e-83a2-ebb451f2b8aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:00:17.925323 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:17.925286    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebbe160-29a5-425e-83a2-ebb451f2b8aa-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:18.535404 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:18.535323    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"
Sep 11 05:00:18.535404 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:18.535351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t" event={"ID":"9ebbe160-29a5-425e-83a2-ebb451f2b8aa","Type":"ContainerDied","Data":"4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb"}
Sep 11 05:00:18.535404 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:18.535399    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bcedff5573f0db0c99c38a15234201a60e7425084a548e3c207f41bab2342cb"
Sep 11 05:00:30.502731 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:30.502701    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"]
Sep 11 05:00:30.505058 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:30.505026    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-89e681d0-29818380-rxbcj"]
Sep 11 05:00:30.745735 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:30.745699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"]
Sep 11 05:00:30.751052 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:30.751023    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-89e681d0-99t7t"]
Sep 11 05:00:32.192427 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:32.192393    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39acb8f8-2e2c-40a4-b566-8c0aa89596ff" path="/var/lib/kubelet/pods/39acb8f8-2e2c-40a4-b566-8c0aa89596ff/volumes"
Sep 11 05:00:32.192832 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:32.192698    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebbe160-29a5-425e-83a2-ebb451f2b8aa" path="/var/lib/kubelet/pods/9ebbe160-29a5-425e-83a2-ebb451f2b8aa/volumes"
Sep 11 05:00:38.761038 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:38.760995    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 05:00:53.780503 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:53.780462    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" podUID="890f3d35-7da0-4d65-832b-79617f1bbeb5" containerName="main" containerID="cri-o://13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d" gracePeriod=45
Sep 11 05:00:54.028700 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.028674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 05:00:54.148654 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148654 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148625    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148713    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njwgn\" (UniqueName: \"kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn\") pod \"890f3d35-7da0-4d65-832b-79617f1bbeb5\" (UID: \"890f3d35-7da0-4d65-832b-79617f1bbeb5\") "
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:00:54.148897 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.148874    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:00:54.149232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.149072    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home" (OuterVolumeSpecName: "home") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:00:54.149232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.149229    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.149301 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.149244    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.149301 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.149253    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.151212 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.151186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn" (OuterVolumeSpecName: "kube-api-access-njwgn") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "kube-api-access-njwgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:00:54.151466 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.151448    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm" (OuterVolumeSpecName: "dshm") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:00:54.151466 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.151453    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "890f3d35-7da0-4d65-832b-79617f1bbeb5" (UID: "890f3d35-7da0-4d65-832b-79617f1bbeb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:00:54.249734 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.249705    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/890f3d35-7da0-4d65-832b-79617f1bbeb5-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.249734 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.249727    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/890f3d35-7da0-4d65-832b-79617f1bbeb5-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.249734 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.249736    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njwgn\" (UniqueName: \"kubernetes.io/projected/890f3d35-7da0-4d65-832b-79617f1bbeb5-kube-api-access-njwgn\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:00:54.750729 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.750691    2581 generic.go:356] "Generic (PLEG): container finished" podID="890f3d35-7da0-4d65-832b-79617f1bbeb5" containerID="13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d" exitCode=0
Sep 11 05:00:54.750906 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.750748    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"
Sep 11 05:00:54.750906 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.750777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" event={"ID":"890f3d35-7da0-4d65-832b-79617f1bbeb5","Type":"ContainerDied","Data":"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"}
Sep 11 05:00:54.750906 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.750821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x" event={"ID":"890f3d35-7da0-4d65-832b-79617f1bbeb5","Type":"ContainerDied","Data":"10f308a3ee28e504ed15154064750f70342ccdd4589545e637fafdbdfd13bfbe"}
Sep 11 05:00:54.750906 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.750836    2581 scope.go:117] "RemoveContainer" containerID="13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"
Sep 11 05:00:54.759853 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.759833    2581 scope.go:117] "RemoveContainer" containerID="13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"
Sep 11 05:00:54.760139 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:00:54.760121    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d\": container with ID starting with 13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d not found: ID does not exist" containerID="13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"
Sep 11 05:00:54.760189 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.760148    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d"} err="failed to get container status \"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d\": rpc error: code = NotFound desc = could not find container \"13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d\": container with ID starting with 13d95ebc90818a7d7e2178766beeea0968777962a4bb33a80556472a04d4af7d not found: ID does not exist"
Sep 11 05:00:54.766325 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.766303    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 05:00:54.768498 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:54.768480    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-89e681d0/e2e-lifecycle-model-89e681d0-kserve-84b5f67cc9-zfg7x"]
Sep 11 05:00:55.286934 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:55.286903    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 05:00:56.192342 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:56.192307    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="890f3d35-7da0-4d65-832b-79617f1bbeb5" path="/var/lib/kubelet/pods/890f3d35-7da0-4d65-832b-79617f1bbeb5/volumes"
Sep 11 05:00:56.363277 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:00:56.363240    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 05:01:06.351425 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:06.351395    2581 scope.go:117] "RemoveContainer" containerID="cbf784db706453f8d25900b2215e8c8fa178a8e71bbe3342a305224fcb2dc8a0"
Sep 11 05:01:06.359966 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:06.359948    2581 scope.go:117] "RemoveContainer" containerID="71fac0a9ed191db46ea7e041a3d61f3ebb623166aba344cd22be5d25a13f20b5"
Sep 11 05:01:10.306921 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.306860    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" podUID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" containerName="main" containerID="cri-o://568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3" gracePeriod=45
Sep 11 05:01:10.542005 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.541979    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 05:01:10.706750 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706727    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.706884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.706884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj68h\" (UniqueName: \"kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.706884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706828    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.706884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706875    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.707098 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.706911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm\") pod \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\" (UID: \"e63ceda0-d391-48ca-86ce-e39d4fcab5b0\") "
Sep 11 05:01:10.707098 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707027    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:10.707210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home" (OuterVolumeSpecName: "home") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:10.707210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:10.707314 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707277    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.707314 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707298    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.707400 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.707313    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.709059 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.709031    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm" (OuterVolumeSpecName: "dshm") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:10.709150 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.709118    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:01:10.709150 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.709125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h" (OuterVolumeSpecName: "kube-api-access-vj68h") pod "e63ceda0-d391-48ca-86ce-e39d4fcab5b0" (UID: "e63ceda0-d391-48ca-86ce-e39d4fcab5b0"). InnerVolumeSpecName "kube-api-access-vj68h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:01:10.808747 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.808715    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.808747 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.808741    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.808747 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.808752    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj68h\" (UniqueName: \"kubernetes.io/projected/e63ceda0-d391-48ca-86ce-e39d4fcab5b0-kube-api-access-vj68h\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:10.842125 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.842089    2581 generic.go:356] "Generic (PLEG): container finished" podID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" containerID="568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3" exitCode=0
Sep 11 05:01:10.842276 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.842135    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"
Sep 11 05:01:10.842276 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.842172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" event={"ID":"e63ceda0-d391-48ca-86ce-e39d4fcab5b0","Type":"ContainerDied","Data":"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"}
Sep 11 05:01:10.842276 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.842224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z" event={"ID":"e63ceda0-d391-48ca-86ce-e39d4fcab5b0","Type":"ContainerDied","Data":"3b029e86f818ec721338fb4d8e16c470de97eff1f56dfec0ff0b78cf226e3303"}
Sep 11 05:01:10.842276 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.842244    2581 scope.go:117] "RemoveContainer" containerID="568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"
Sep 11 05:01:10.851565 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.851547    2581 scope.go:117] "RemoveContainer" containerID="568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"
Sep 11 05:01:10.851883 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:01:10.851864    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3\": container with ID starting with 568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3 not found: ID does not exist" containerID="568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"
Sep 11 05:01:10.851951 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.851896    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3"} err="failed to get container status \"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3\": rpc error: code = NotFound desc = could not find container \"568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3\": container with ID starting with 568eb882cb95acd5daa648c2d15ee8553b43ba17f5b0a2b44a3da9dabd2580c3 not found: ID does not exist"
Sep 11 05:01:10.862863 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.862836    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 05:01:10.869791 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:10.869762    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/auth-test-model-c1858a-kserve-85c89bf655-2764z"]
Sep 11 05:01:11.384005 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.383968    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" podUID="d0902714-6744-42cf-829c-2215ff08e8ba" containerName="main" containerID="cri-o://e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721" gracePeriod=45
Sep 11 05:01:11.633907 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.633883    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 05:01:11.817095 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcpv2\" (UniqueName: \"kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817095 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817335 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817119    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817335 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817335 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817232    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817335 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm\") pod \"d0902714-6744-42cf-829c-2215ff08e8ba\" (UID: \"d0902714-6744-42cf-829c-2215ff08e8ba\") "
Sep 11 05:01:11.817520 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817343    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home" (OuterVolumeSpecName: "home") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:11.817520 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817416    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:11.817520 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817489    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:11.817732 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817663    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:11.817732 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817679    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:11.817732 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.817689    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:11.819498 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.819475    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm" (OuterVolumeSpecName: "dshm") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:01:11.819614 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.819599    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2" (OuterVolumeSpecName: "kube-api-access-qcpv2") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "kube-api-access-qcpv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:01:11.819659 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.819606    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0902714-6744-42cf-829c-2215ff08e8ba" (UID: "d0902714-6744-42cf-829c-2215ff08e8ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:01:11.848570 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.848541    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0902714-6744-42cf-829c-2215ff08e8ba" containerID="e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721" exitCode=0
Sep 11 05:01:11.848715 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.848610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" event={"ID":"d0902714-6744-42cf-829c-2215ff08e8ba","Type":"ContainerDied","Data":"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"}
Sep 11 05:01:11.848715 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.848663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs" event={"ID":"d0902714-6744-42cf-829c-2215ff08e8ba","Type":"ContainerDied","Data":"91654945abbddd1479514040f8f1b13bb61af0a2ec013f38b33ada20496982cf"}
Sep 11 05:01:11.848715 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.848626    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"
Sep 11 05:01:11.848715 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.848698    2581 scope.go:117] "RemoveContainer" containerID="e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"
Sep 11 05:01:11.857916 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.857898    2581 scope.go:117] "RemoveContainer" containerID="e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"
Sep 11 05:01:11.858172 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:01:11.858153    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721\": container with ID starting with e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721 not found: ID does not exist" containerID="e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"
Sep 11 05:01:11.858235 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.858180    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721"} err="failed to get container status \"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721\": rpc error: code = NotFound desc = could not find container \"e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721\": container with ID starting with e68abbaa0ee5fc9b797287eca7a826f0f2c482a38c0d9d4f6464d0b4e94c3721 not found: ID does not exist"
Sep 11 05:01:11.870519 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.870499    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 05:01:11.874368 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.874348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/auth-test-model-16b799-kserve-f585f647f-8t6qs"]
Sep 11 05:01:11.918684 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.918660    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcpv2\" (UniqueName: \"kubernetes.io/projected/d0902714-6744-42cf-829c-2215ff08e8ba-kube-api-access-qcpv2\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:11.918684 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.918685    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0902714-6744-42cf-829c-2215ff08e8ba-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:11.918829 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:11.918694    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0902714-6744-42cf-829c-2215ff08e8ba-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:01:12.192757 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:12.192687    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0902714-6744-42cf-829c-2215ff08e8ba" path="/var/lib/kubelet/pods/d0902714-6744-42cf-829c-2215ff08e8ba/volumes"
Sep 11 05:01:12.193057 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:12.193044    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" path="/var/lib/kubelet/pods/e63ceda0-d391-48ca-86ce-e39d4fcab5b0/volumes"
Sep 11 05:01:19.260281 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260251    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260624    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" containerName="main"
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260638    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" containerName="main"
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260655    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890f3d35-7da0-4d65-832b-79617f1bbeb5" containerName="main"
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260660    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="890f3d35-7da0-4d65-832b-79617f1bbeb5" containerName="main"
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260668    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0902714-6744-42cf-829c-2215ff08e8ba" containerName="main"
Sep 11 05:01:19.260673 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260675    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0902714-6744-42cf-829c-2215ff08e8ba" containerName="main"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260681    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebbe160-29a5-425e-83a2-ebb451f2b8aa" containerName="revoke-keys"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260687    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebbe160-29a5-425e-83a2-ebb451f2b8aa" containerName="revoke-keys"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260742    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0902714-6744-42cf-829c-2215ff08e8ba" containerName="main"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260751    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebbe160-29a5-425e-83a2-ebb451f2b8aa" containerName="revoke-keys"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260761    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e63ceda0-d391-48ca-86ce-e39d4fcab5b0" containerName="main"
Sep 11 05:01:19.260877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.260766    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="890f3d35-7da0-4d65-832b-79617f1bbeb5" containerName="main"
Sep 11 05:01:19.264993 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.264975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.267297 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.267281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"kube-root-ca.crt\""
Sep 11 05:01:19.267838 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.267818    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"openshift-service-ca.crt\""
Sep 11 05:01:19.267943 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.267828    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"default-dockercfg-lxgst\""
Sep 11 05:01:19.268033 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.268015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"sub-test-model-c1858a-kserve-self-signed-certs\""
Sep 11 05:01:19.275416 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.275396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:01:19.277214 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.277369 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277239    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.277369 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.277369 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdpf5\" (UniqueName: \"kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.277630 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.277792 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.277766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378357 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378572 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378572 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378572 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdpf5\" (UniqueName: \"kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378572 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378854 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.378854 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.378816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.379166 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.379112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.379302 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.379257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.381161 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.381138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.381803 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.381782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.386921 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.386901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdpf5\" (UniqueName: \"kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5\") pod \"sub-test-model-c1858a-kserve-5857cb48c-ddblm\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.574438 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.574358    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:19.846740 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.846715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:01:19.848623 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:01:19.848572    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod217941e6_d8b7_4f55_8431_edf85016ad6a.slice/crio-71f29e6b30b6d8f06901b76809260119857396c3e67c25dc9aaed0d712bac5b0 WatchSource:0}: Error finding container 71f29e6b30b6d8f06901b76809260119857396c3e67c25dc9aaed0d712bac5b0: Status 404 returned error can't find the container with id 71f29e6b30b6d8f06901b76809260119857396c3e67c25dc9aaed0d712bac5b0
Sep 11 05:01:19.894189 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:19.894162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" event={"ID":"217941e6-d8b7-4f55-8431-edf85016ad6a","Type":"ContainerStarted","Data":"71f29e6b30b6d8f06901b76809260119857396c3e67c25dc9aaed0d712bac5b0"}
Sep 11 05:01:20.901331 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:20.901290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" event={"ID":"217941e6-d8b7-4f55-8431-edf85016ad6a","Type":"ContainerStarted","Data":"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"}
Sep 11 05:01:20.918861 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:20.918810    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" podStartSLOduration=1.9187945979999999 podStartE2EDuration="1.918794598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:01:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:01:20.917644865 +0000 UTC m=+1275.209119303" watchObservedRunningTime="2026-09-11 05:01:20.918794598 +0000 UTC m=+1275.210269035"
Sep 11 05:01:29.575005 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:29.574971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:29.575482 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:29.575044    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:29.587588 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:29.587561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:29.961420 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:29.961386    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:01:46.962448 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.962411    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:01:46.965807 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.965787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:46.968559 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.968536    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"kube-root-ca.crt\""
Sep 11 05:01:46.969391 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.969374    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"openshift-service-ca.crt\""
Sep 11 05:01:46.969499 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.969443    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"default-dockercfg-vwxtm\""
Sep 11 05:01:46.969570 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.969515    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"sub-test-model-16b799-kserve-self-signed-certs\""
Sep 11 05:01:46.983401 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:46.983380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:01:47.116359 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.116530 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.116530 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.116530 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.116739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ftxl\" (UniqueName: \"kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.116739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.116650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218086 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218086 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218086 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ftxl\" (UniqueName: \"kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218086 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218531 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218531 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.218631 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.218538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.220566 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.220547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.220786 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.220770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.225862 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.225845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ftxl\" (UniqueName: \"kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl\") pod \"sub-test-model-16b799-kserve-698584896c-7f869\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") " pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.275926 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.275896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:47.549352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:47.549323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:01:47.551755 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:01:47.551729    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6e1a6bc_b021_4b5e_a912_b62dc7bd9ced.slice/crio-5660882ca09fece1b05c6343e5a10e36965ee93ba44b37ac14a57041a28d9584 WatchSource:0}: Error finding container 5660882ca09fece1b05c6343e5a10e36965ee93ba44b37ac14a57041a28d9584: Status 404 returned error can't find the container with id 5660882ca09fece1b05c6343e5a10e36965ee93ba44b37ac14a57041a28d9584
Sep 11 05:01:48.056982 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:48.056946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" event={"ID":"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced","Type":"ContainerStarted","Data":"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"}
Sep 11 05:01:48.056982 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:48.056989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" event={"ID":"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced","Type":"ContainerStarted","Data":"5660882ca09fece1b05c6343e5a10e36965ee93ba44b37ac14a57041a28d9584"}
Sep 11 05:01:48.079363 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:48.079309    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" podStartSLOduration=2.079292986 podStartE2EDuration="2.079292986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:01:48.077290991 +0000 UTC m=+1302.368765428" watchObservedRunningTime="2026-09-11 05:01:48.079292986 +0000 UTC m=+1302.370767795"
Sep 11 05:01:57.276795 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:57.276760    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:57.277220 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:57.276839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:57.289385 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:57.289364    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:01:58.128674 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:01:58.128647    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:02:06.389972 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:06.389942    2581 scope.go:117] "RemoveContainer" containerID="2fb8c4016de5d9b5f11ce3bc8b62177464409f856b79f8dff93eb1655d858599"
Sep 11 05:02:28.686178 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.686107    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7518-2e8fc9"]
Sep 11 05:02:28.691226 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.691210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:28.693932 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.693909    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mgwvk\""
Sep 11 05:02:28.695662 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.695639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7518-2e8fc9"]
Sep 11 05:02:28.773397 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.773367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t24kn\" (UniqueName: \"kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn\") pod \"test-curl-7518-2e8fc9\" (UID: \"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68\") " pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:28.874283 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.874258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t24kn\" (UniqueName: \"kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn\") pod \"test-curl-7518-2e8fc9\" (UID: \"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68\") " pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:28.881981 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:28.881963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t24kn\" (UniqueName: \"kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn\") pod \"test-curl-7518-2e8fc9\" (UID: \"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68\") " pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:29.001853 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:29.001830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:29.276211 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:29.270760    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7518-2e8fc9"]
Sep 11 05:02:29.284334 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:29.284312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7518-2e8fc9" event={"ID":"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68","Type":"ContainerStarted","Data":"30975db68e7cc346ea1450e78eb09c5d4b818f23589c04d03e1c65c3d79d2208"}
Sep 11 05:02:30.291326 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:30.291291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7518-2e8fc9" event={"ID":"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68","Type":"ContainerStarted","Data":"fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b"}
Sep 11 05:02:30.307831 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:30.307785    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7518-2e8fc9" podStartSLOduration=2.307771635 podStartE2EDuration="2.307771635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:30.306377845 +0000 UTC m=+1344.597852282" watchObservedRunningTime="2026-09-11 05:02:30.307771635 +0000 UTC m=+1344.599246073"
Sep 11 05:02:30.890270 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:30.890234    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7518-2e8fc9"]
Sep 11 05:02:30.894390 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:30.894363    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7518-2e8fc9"]
Sep 11 05:02:31.006075 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.006049    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7518-174a64"]
Sep 11 05:02:31.006412 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.006399    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" containerName="test-curl-7518-2e8fc9"
Sep 11 05:02:31.006457 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.006414    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" containerName="test-curl-7518-2e8fc9"
Sep 11 05:02:31.006492 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.006482    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" containerName="test-curl-7518-2e8fc9"
Sep 11 05:02:31.009530 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.009515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:31.015244 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.015219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7518-174a64"]
Sep 11 05:02:31.092300 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.092276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jj8f\" (UniqueName: \"kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f\") pod \"test-curl-7518-174a64\" (UID: \"34356554-199c-49e0-8abc-411d5a5377a1\") " pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:31.193635 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.193608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jj8f\" (UniqueName: \"kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f\") pod \"test-curl-7518-174a64\" (UID: \"34356554-199c-49e0-8abc-411d5a5377a1\") " pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:31.203242 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.203221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jj8f\" (UniqueName: \"kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f\") pod \"test-curl-7518-174a64\" (UID: \"34356554-199c-49e0-8abc-411d5a5377a1\") " pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:31.297357 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:31.297297    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7518-2e8fc9" container="test-curl-7518-2e8fc9"
Sep 11 05:02:31.320210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.320190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:31.589531 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:31.589502    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7518-174a64"]
Sep 11 05:02:31.590454 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:02:31.590430    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34356554_199c_49e0_8abc_411d5a5377a1.slice/crio-dd523941120903d15fd6d9b91d553adac1c0b9aa2c5ac6abd7cd27888d40ce58 WatchSource:0}: Error finding container dd523941120903d15fd6d9b91d553adac1c0b9aa2c5ac6abd7cd27888d40ce58: Status 404 returned error can't find the container with id dd523941120903d15fd6d9b91d553adac1c0b9aa2c5ac6abd7cd27888d40ce58
Sep 11 05:02:32.303612 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.303562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7518-174a64" event={"ID":"34356554-199c-49e0-8abc-411d5a5377a1","Type":"ContainerStarted","Data":"bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7"}
Sep 11 05:02:32.304047 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.303619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7518-174a64" event={"ID":"34356554-199c-49e0-8abc-411d5a5377a1","Type":"ContainerStarted","Data":"dd523941120903d15fd6d9b91d553adac1c0b9aa2c5ac6abd7cd27888d40ce58"}
Sep 11 05:02:32.304047 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.303677    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7518-2e8fc9" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" containerName="test-curl-7518-2e8fc9" containerID="cri-o://fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b" gracePeriod=2
Sep 11 05:02:32.319804 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.319762    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7518-174a64" podStartSLOduration=2.319749624 podStartE2EDuration="2.319749624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:32.317215383 +0000 UTC m=+1346.608689801" watchObservedRunningTime="2026-09-11 05:02:32.319749624 +0000 UTC m=+1346.611224061"
Sep 11 05:02:32.924693 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.924654    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7518-174a64"]
Sep 11 05:02:32.930761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:32.930730    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7518-174a64"]
Sep 11 05:02:33.309054 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:33.309028    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7518-174a64" container="test-curl-7518-174a64"
Sep 11 05:02:34.314835 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.314777    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7518-174a64" podUID="34356554-199c-49e0-8abc-411d5a5377a1" containerName="test-curl-7518-174a64" containerID="cri-o://bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7" gracePeriod=2
Sep 11 05:02:34.485303 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.485279    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:34.519820 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.519799    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t24kn\" (UniqueName: \"kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn\") pod \"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68\" (UID: \"0e82c0b8-8d23-49e5-99e5-2bd7c3568a68\") "
Sep 11 05:02:34.521805 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.521784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn" (OuterVolumeSpecName: "kube-api-access-t24kn") pod "0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" (UID: "0e82c0b8-8d23-49e5-99e5-2bd7c3568a68"). InnerVolumeSpecName "kube-api-access-t24kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:02:34.620909 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.620856    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t24kn\" (UniqueName: \"kubernetes.io/projected/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68-kube-api-access-t24kn\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:34.965465 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:34.965437    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:02:35.321336 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.321256    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" containerID="fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b" exitCode=137
Sep 11 05:02:35.321336 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.321291    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:35.321801 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.321348    2581 scope.go:117] "RemoveContainer" containerID="fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b"
Sep 11 05:02:35.330482 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.330469    2581 scope.go:117] "RemoveContainer" containerID="fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b"
Sep 11 05:02:35.330772 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:35.330738    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b\": container with ID starting with fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b not found: ID does not exist" containerID="fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b"
Sep 11 05:02:35.330825 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.330779    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b"} err="failed to get container status \"fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b\": rpc error: code = NotFound desc = could not find container \"fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b\": container with ID starting with fae405fc8f4965cbfed29921cfb10e850430b05186495732c5bbeecd8c26e42b not found: ID does not exist"
Sep 11 05:02:35.331105 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:35.331084    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7518-2e8fc9\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:35.952986 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:35.952953    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:02:35.955151 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:35.955112    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7518-2e8fc9\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:36.191030 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:36.190992    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7518-2e8fc9\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" pod="opendatahub/test-curl-7518-2e8fc9"
Sep 11 05:02:36.192438 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:36.192417    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e82c0b8-8d23-49e5-99e5-2bd7c3568a68" path="/var/lib/kubelet/pods/0e82c0b8-8d23-49e5-99e5-2bd7c3568a68/volumes"
Sep 11 05:02:36.456009 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:36.455988    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:36.537248 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:36.537227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jj8f\" (UniqueName: \"kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f\") pod \"34356554-199c-49e0-8abc-411d5a5377a1\" (UID: \"34356554-199c-49e0-8abc-411d5a5377a1\") "
Sep 11 05:02:36.539373 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:36.539348    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f" (OuterVolumeSpecName: "kube-api-access-6jj8f") pod "34356554-199c-49e0-8abc-411d5a5377a1" (UID: "34356554-199c-49e0-8abc-411d5a5377a1"). InnerVolumeSpecName "kube-api-access-6jj8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:02:36.638136 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:36.638079    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jj8f\" (UniqueName: \"kubernetes.io/projected/34356554-199c-49e0-8abc-411d5a5377a1-kube-api-access-6jj8f\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:37.334770 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:37.334736    2581 generic.go:356] "Generic (PLEG): container finished" podID="34356554-199c-49e0-8abc-411d5a5377a1" containerID="bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7" exitCode=137
Sep 11 05:02:37.334963 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:37.334784    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:37.334963 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:37.334830    2581 scope.go:117] "RemoveContainer" containerID="bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7"
Sep 11 05:02:37.344716 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:37.344701    2581 scope.go:117] "RemoveContainer" containerID="bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7"
Sep 11 05:02:37.344948 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:37.344931    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7\": container with ID starting with bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7 not found: ID does not exist" containerID="bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7"
Sep 11 05:02:37.345015 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:37.344960    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7"} err="failed to get container status \"bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7\": rpc error: code = NotFound desc = could not find container \"bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7\": container with ID starting with bd92cfb6dae5144cc38340499b5f3ec938de1fec5c8dbdea91cd72a1596363b7 not found: ID does not exist"
Sep 11 05:02:37.345569 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:37.345549    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7518-174a64\" is forbidden: User \"system:node:ip-10-0-139-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-137.ec2.internal' and this object" podUID="34356554-199c-49e0-8abc-411d5a5377a1" pod="opendatahub/test-curl-7518-174a64"
Sep 11 05:02:38.193553 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:38.193518    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34356554-199c-49e0-8abc-411d5a5377a1" path="/var/lib/kubelet/pods/34356554-199c-49e0-8abc-411d5a5377a1/volumes"
Sep 11 05:02:44.258414 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.258385    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:02:44.258812 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.258736    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34356554-199c-49e0-8abc-411d5a5377a1" containerName="test-curl-7518-174a64"
Sep 11 05:02:44.258812 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.258747    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="34356554-199c-49e0-8abc-411d5a5377a1" containerName="test-curl-7518-174a64"
Sep 11 05:02:44.258887 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.258814    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="34356554-199c-49e0-8abc-411d5a5377a1" containerName="test-curl-7518-174a64"
Sep 11 05:02:44.263201 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.263185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.265470 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.265449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c1858a\"/\"rate-test-model-c1858a-kserve-self-signed-certs\""
Sep 11 05:02:44.270309 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.270285    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:02:44.302946 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.302927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.303076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.302966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czbkv\" (UniqueName: \"kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.303076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.302989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.303076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.303021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.303076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.303069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.303237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.303116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.403950 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.403923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404113 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.403973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czbkv\" (UniqueName: \"kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404113 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404228 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404285 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404285 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404402 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404491 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.404561 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.404547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.406422 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.406402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.406667 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.406651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.412056 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.412035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czbkv\" (UniqueName: \"kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv\") pod \"rate-test-model-c1858a-kserve-645bd9859f-xgw4b\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") " pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.575130 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.575046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:44.849981 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:02:44.849949    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod258d5c5d_ad68_44fa_be11_ed289e92c6d6.slice/crio-d32d4ed860862da9309ca3151d6fe75928b9441f59871140bfb7a25a7f86d9db WatchSource:0}: Error finding container d32d4ed860862da9309ca3151d6fe75928b9441f59871140bfb7a25a7f86d9db: Status 404 returned error can't find the container with id d32d4ed860862da9309ca3151d6fe75928b9441f59871140bfb7a25a7f86d9db
Sep 11 05:02:44.851117 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:44.851097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:02:45.381885 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.381846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" event={"ID":"258d5c5d-ad68-44fa-be11-ed289e92c6d6","Type":"ContainerStarted","Data":"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"}
Sep 11 05:02:45.382362 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.381902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" event={"ID":"258d5c5d-ad68-44fa-be11-ed289e92c6d6","Type":"ContainerStarted","Data":"d32d4ed860862da9309ca3151d6fe75928b9441f59871140bfb7a25a7f86d9db"}
Sep 11 05:02:45.399852 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.399820    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:02:45.403382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.403365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.405663 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.405643    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9c1509-openshift-default-dockercfg-vk92h\""
Sep 11 05:02:45.406676 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.406642    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" podStartSLOduration=1.406629863 podStartE2EDuration="1.406629863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:45.404372215 +0000 UTC m=+1359.695846655" watchObservedRunningTime="2026-09-11 05:02:45.406629863 +0000 UTC m=+1359.698104299"
Sep 11 05:02:45.417797 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.417765    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:02:45.515011 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.514988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zkql\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515135 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515135 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515135 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515135 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515327 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515327 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515327 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.515327 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.515263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.616838 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zkql\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.616838 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.616991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617456 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617456 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617456 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617634 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.617824 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.617805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.619505 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.619486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.619729 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.619712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.626785 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.626760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.627682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.627663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zkql\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql\") pod \"e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") " pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:45.724942 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:45.724920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:46.000188 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.000159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:02:46.001625 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:02:46.001601    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0519827c_f2ef_4f68_920d_de13080e1a25.slice/crio-91881e8a5cf69acafb672df30bd19627e7f315c67e0582463c4fe90ce871320a WatchSource:0}: Error finding container 91881e8a5cf69acafb672df30bd19627e7f315c67e0582463c4fe90ce871320a: Status 404 returned error can't find the container with id 91881e8a5cf69acafb672df30bd19627e7f315c67e0582463c4fe90ce871320a
Sep 11 05:02:46.003914 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.003886    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:02:46.003992 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.003949    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:02:46.003992 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.003982    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:02:46.389101 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.389015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerStarted","Data":"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"}
Sep 11 05:02:46.389101 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.389074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerStarted","Data":"91881e8a5cf69acafb672df30bd19627e7f315c67e0582463c4fe90ce871320a"}
Sep 11 05:02:46.423168 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.423118    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podStartSLOduration=1.423099692 podStartE2EDuration="1.423099692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:46.418456299 +0000 UTC m=+1360.709930736" watchObservedRunningTime="2026-09-11 05:02:46.423099692 +0000 UTC m=+1360.714574127"
Sep 11 05:02:46.725829 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:46.725803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:47.731082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:47.731056    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:48.401041 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:48.401009    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:48.401970 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:48.401950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:02:49.984192 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:49.984150    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" podUID="217941e6-d8b7-4f55-8431-edf85016ad6a" containerName="main" containerID="cri-o://31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664" gracePeriod=45
Sep 11 05:02:50.229408 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.229387    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:02:50.358048 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.357982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358048 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358037    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358107    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdpf5\" (UniqueName: \"kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5\") pod \"217941e6-d8b7-4f55-8431-edf85016ad6a\" (UID: \"217941e6-d8b7-4f55-8431-edf85016ad6a\") "
Sep 11 05:02:50.358445 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358233    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home" (OuterVolumeSpecName: "home") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:50.358445 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358247    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache" (OuterVolumeSpecName: "model-cache") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:50.358445 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:50.358568 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358504    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.358568 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358524    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.358568 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.358532    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.360377 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.360355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:02:50.360502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.360386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm" (OuterVolumeSpecName: "dshm") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:50.360502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.360474    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5" (OuterVolumeSpecName: "kube-api-access-wdpf5") pod "217941e6-d8b7-4f55-8431-edf85016ad6a" (UID: "217941e6-d8b7-4f55-8431-edf85016ad6a"). InnerVolumeSpecName "kube-api-access-wdpf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:02:50.417003 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.416972    2581 generic.go:356] "Generic (PLEG): container finished" podID="217941e6-d8b7-4f55-8431-edf85016ad6a" containerID="31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664" exitCode=0
Sep 11 05:02:50.417154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.417027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"
Sep 11 05:02:50.417154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.417051    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" event={"ID":"217941e6-d8b7-4f55-8431-edf85016ad6a","Type":"ContainerDied","Data":"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"}
Sep 11 05:02:50.417154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.417097    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm" event={"ID":"217941e6-d8b7-4f55-8431-edf85016ad6a","Type":"ContainerDied","Data":"71f29e6b30b6d8f06901b76809260119857396c3e67c25dc9aaed0d712bac5b0"}
Sep 11 05:02:50.417154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.417120    2581 scope.go:117] "RemoveContainer" containerID="31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"
Sep 11 05:02:50.426738 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.426727    2581 scope.go:117] "RemoveContainer" containerID="31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"
Sep 11 05:02:50.427130 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:50.427024    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664\": container with ID starting with 31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664 not found: ID does not exist" containerID="31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"
Sep 11 05:02:50.427130 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.427060    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664"} err="failed to get container status \"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664\": rpc error: code = NotFound desc = could not find container \"31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664\": container with ID starting with 31ee3a2bf275239d60ca0cfff777995476eda21a13fec9d4c216c2fbb534e664 not found: ID does not exist"
Sep 11 05:02:50.443539 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.443520    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:02:50.450485 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.450465    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/sub-test-model-c1858a-kserve-5857cb48c-ddblm"]
Sep 11 05:02:50.459527 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.459506    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdpf5\" (UniqueName: \"kubernetes.io/projected/217941e6-d8b7-4f55-8431-edf85016ad6a-kube-api-access-wdpf5\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.459632 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.459533    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/217941e6-d8b7-4f55-8431-edf85016ad6a-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.459632 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.459547    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217941e6-d8b7-4f55-8431-edf85016ad6a-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:50.972033 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:50.971986    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" podUID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" containerName="main" containerID="cri-o://a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784" gracePeriod=45
Sep 11 05:02:51.214647 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.214619    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:02:51.267803 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267747    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.267922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267800    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.267922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267825    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ftxl\" (UniqueName: \"kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.267922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.267922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267880    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.267922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.267914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm\") pod \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\" (UID: \"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced\") "
Sep 11 05:02:51.268200 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268048    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:51.268200 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home" (OuterVolumeSpecName: "home") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:51.268200 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268151    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:51.268833 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268663    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.268833 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268689    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.268833 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.268701    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.270138 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.270111    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:02:51.270241 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.270134    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl" (OuterVolumeSpecName: "kube-api-access-4ftxl") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "kube-api-access-4ftxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:02:51.270241 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.270140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm" (OuterVolumeSpecName: "dshm") pod "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" (UID: "f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:02:51.369560 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.369537    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.369560 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.369557    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ftxl\" (UniqueName: \"kubernetes.io/projected/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-kube-api-access-4ftxl\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.369705 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.369566    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:02:51.423651 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.423623    2581 generic.go:356] "Generic (PLEG): container finished" podID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" containerID="a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784" exitCode=0
Sep 11 05:02:51.423743 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.423665    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"
Sep 11 05:02:51.423743 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.423706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" event={"ID":"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced","Type":"ContainerDied","Data":"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"}
Sep 11 05:02:51.423851 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.423748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869" event={"ID":"f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced","Type":"ContainerDied","Data":"5660882ca09fece1b05c6343e5a10e36965ee93ba44b37ac14a57041a28d9584"}
Sep 11 05:02:51.423851 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.423772    2581 scope.go:117] "RemoveContainer" containerID="a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"
Sep 11 05:02:51.433011 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.432996    2581 scope.go:117] "RemoveContainer" containerID="a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"
Sep 11 05:02:51.433294 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:02:51.433275    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784\": container with ID starting with a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784 not found: ID does not exist" containerID="a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"
Sep 11 05:02:51.433350 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.433302    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784"} err="failed to get container status \"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784\": rpc error: code = NotFound desc = could not find container \"a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784\": container with ID starting with a438243694c2984390dabad921753497dc3610f43b6b05a7f091f7fd162a0784 not found: ID does not exist"
Sep 11 05:02:51.445033 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.445011    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:02:51.449498 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:51.449479    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/sub-test-model-16b799-kserve-698584896c-7f869"]
Sep 11 05:02:52.197599 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:52.193844    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="217941e6-d8b7-4f55-8431-edf85016ad6a" path="/var/lib/kubelet/pods/217941e6-d8b7-4f55-8431-edf85016ad6a/volumes"
Sep 11 05:02:52.197599 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:52.194417    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" path="/var/lib/kubelet/pods/f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced/volumes"
Sep 11 05:02:54.575258 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:54.575222    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:54.575690 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:54.575274    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:54.587750 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:54.587730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:02:55.459548 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:02:55.459519    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:03:08.524194 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.524159    2581 generic.go:356] "Generic (PLEG): container finished" podID="0519827c-f2ef-4f68-920d-de13080e1a25" containerID="4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b" exitCode=0
Sep 11 05:03:08.524627 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.524230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerDied","Data":"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"}
Sep 11 05:03:08.524680 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.524653    2581 scope.go:117] "RemoveContainer" containerID="4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"
Sep 11 05:03:08.525772 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.525749    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:03:08.525831 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.525812    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:03:08.525866 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:08.525842    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:03:09.536670 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:09.536630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerStarted","Data":"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"}
Sep 11 05:03:09.725174 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:09.725134    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:03:09.725502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:09.725463    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:09.725605 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:09.725542    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:10.725926 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:10.725882    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:10.726436 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:10.725951    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:11.726308 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:11.726264    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:11.726835 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:11.726334    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:12.725385 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:12.725349    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:12.725698 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:12.725666    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:13.726328 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:13.726288    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:13.726835 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:13.726351    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:14.725895 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:14.725853    2581 patch_prober.go:28] interesting pod/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 11 05:03:14.726107 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:14.725928    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 11 05:03:15.729758 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:15.729728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:03:15.730231 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:15.730091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:03:15.730780 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:15.730762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:03:16.135614 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.135494    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"]
Sep 11 05:03:16.135945 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.135920    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" containerName="main"
Sep 11 05:03:16.135945 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.135942    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" containerName="main"
Sep 11 05:03:16.136115 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.135970    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="217941e6-d8b7-4f55-8431-edf85016ad6a" containerName="main"
Sep 11 05:03:16.136115 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.135976    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="217941e6-d8b7-4f55-8431-edf85016ad6a" containerName="main"
Sep 11 05:03:16.136115 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.136035    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e1a6bc-b021-4b5e-a912-b62dc7bd9ced" containerName="main"
Sep 11 05:03:16.136115 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.136043    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="217941e6-d8b7-4f55-8431-edf85016ad6a" containerName="main"
Sep 11 05:03:16.137974 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.137959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:16.140116 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.140100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:03:16.140207 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.140103    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:03:16.148679 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.148648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"]
Sep 11 05:03:16.199596 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.199544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b9198b-drpl9\" (UID: \"b5411938-b13d-462c-ab66-6141e6606748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:16.300886 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.300860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b9198b-drpl9\" (UID: \"b5411938-b13d-462c-ab66-6141e6606748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:16.301401 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.301384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b9198b-drpl9\" (UID: \"b5411938-b13d-462c-ab66-6141e6606748\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:16.452258 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.452236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:16.722481 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:16.722456    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"]
Sep 11 05:03:16.724354 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:03:16.724324    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5411938_b13d_462c_ab66_6141e6606748.slice/crio-797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f WatchSource:0}: Error finding container 797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f: Status 404 returned error can't find the container with id 797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f
Sep 11 05:03:17.587210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:17.587182    2581 generic.go:356] "Generic (PLEG): container finished" podID="b5411938-b13d-462c-ab66-6141e6606748" containerID="e554685f135c8c30ecd3bab3f3913c0f81e901292a2e57f0183e6f58d9d94a83" exitCode=0
Sep 11 05:03:17.587552 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:17.587261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9" event={"ID":"b5411938-b13d-462c-ab66-6141e6606748","Type":"ContainerDied","Data":"e554685f135c8c30ecd3bab3f3913c0f81e901292a2e57f0183e6f58d9d94a83"}
Sep 11 05:03:17.587552 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:17.587310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9" event={"ID":"b5411938-b13d-462c-ab66-6141e6606748","Type":"ContainerStarted","Data":"797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f"}
Sep 11 05:03:18.724177 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:18.724151    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:18.823650 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:18.823620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca\") pod \"b5411938-b13d-462c-ab66-6141e6606748\" (UID: \"b5411938-b13d-462c-ab66-6141e6606748\") "
Sep 11 05:03:18.823952 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:18.823932    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b5411938-b13d-462c-ab66-6141e6606748" (UID: "b5411938-b13d-462c-ab66-6141e6606748"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:03:18.924637 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:18.924544    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5411938-b13d-462c-ab66-6141e6606748-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:03:19.600655 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:19.600627    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"
Sep 11 05:03:19.600880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:19.600623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9" event={"ID":"b5411938-b13d-462c-ab66-6141e6606748","Type":"ContainerDied","Data":"797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f"}
Sep 11 05:03:19.600880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:19.600751    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="797168955b9327c4d967020597735fca704e5437724162d546eb0729161a8c9f"
Sep 11 05:03:31.563907 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.563875    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:03:31.564352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.564194    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5411938-b13d-462c-ab66-6141e6606748" containerName="revoke-keys"
Sep 11 05:03:31.564352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.564205    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5411938-b13d-462c-ab66-6141e6606748" containerName="revoke-keys"
Sep 11 05:03:31.564352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.564293    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5411938-b13d-462c-ab66-6141e6606748" containerName="revoke-keys"
Sep 11 05:03:31.566775 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.566756    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.569892 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.569865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"kube-root-ca.crt\""
Sep 11 05:03:31.569892 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.569887    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"rate-test-model-16b799-kserve-self-signed-certs\""
Sep 11 05:03:31.570074 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.569890    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"default-dockercfg-vwxtm\""
Sep 11 05:03:31.570074 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.569934    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-16b799\"/\"openshift-service-ca.crt\""
Sep 11 05:03:31.578315 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.578291    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:03:31.633053 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.633215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.633215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.633215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.633215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scghr\" (UniqueName: \"kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.633374 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.633225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734522 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scghr\" (UniqueName: \"kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.734720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.735007 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.735007 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.734933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.735096 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.735077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.737034 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.737010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.737441 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.737422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.743047 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.743027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scghr\" (UniqueName: \"kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr\") pod \"rate-test-model-16b799-kserve-6f6ff664c-skzh7\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") " pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:31.856226 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.856157    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"]
Sep 11 05:03:31.860720 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.860699    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b9198b-drpl9"]
Sep 11 05:03:31.876231 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:31.876213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:32.150483 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:32.150459    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:03:32.151696 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:03:32.151668    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52055a68_d0b7_426b_b193_6971ae7c34ca.slice/crio-9ac86c16df17e5210b7fd3c7ba5dfcaef53f60ab2dba7879dd5ad132068b4e15 WatchSource:0}: Error finding container 9ac86c16df17e5210b7fd3c7ba5dfcaef53f60ab2dba7879dd5ad132068b4e15: Status 404 returned error can't find the container with id 9ac86c16df17e5210b7fd3c7ba5dfcaef53f60ab2dba7879dd5ad132068b4e15
Sep 11 05:03:32.192330 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:32.192308    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5411938-b13d-462c-ab66-6141e6606748" path="/var/lib/kubelet/pods/b5411938-b13d-462c-ab66-6141e6606748/volumes"
Sep 11 05:03:32.673668 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:32.673633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" event={"ID":"52055a68-d0b7-426b-b193-6971ae7c34ca","Type":"ContainerStarted","Data":"2cdbf1393aab91574c54cd2bd403d14a311ac6f8b83f52033275433460e41c1b"}
Sep 11 05:03:32.674054 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:32.673675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" event={"ID":"52055a68-d0b7-426b-b193-6971ae7c34ca","Type":"ContainerStarted","Data":"9ac86c16df17e5210b7fd3c7ba5dfcaef53f60ab2dba7879dd5ad132068b4e15"}
Sep 11 05:03:32.690049 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:32.690005    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" podStartSLOduration=1.6899902839999998 podStartE2EDuration="1.689990284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:03:32.689313578 +0000 UTC m=+1406.980788015" watchObservedRunningTime="2026-09-11 05:03:32.689990284 +0000 UTC m=+1406.981464738"
Sep 11 05:03:39.334635 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.334603    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"]
Sep 11 05:03:39.337850 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.337835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:39.340029 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.340007    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:03:39.340154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.340087    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:03:39.346027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.345933    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"]
Sep 11 05:03:39.404029 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.403998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr\" (UID: \"712ed70b-7ff9-4dfa-bd65-985ccf82f430\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:39.504525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.504492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr\" (UID: \"712ed70b-7ff9-4dfa-bd65-985ccf82f430\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:39.505100 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.505081    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr\" (UID: \"712ed70b-7ff9-4dfa-bd65-985ccf82f430\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:39.650031 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.649949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:39.925969 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:39.925941    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"]
Sep 11 05:03:39.926502 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:03:39.926477    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod712ed70b_7ff9_4dfa_bd65_985ccf82f430.slice/crio-0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04 WatchSource:0}: Error finding container 0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04: Status 404 returned error can't find the container with id 0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04
Sep 11 05:03:40.724226 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:40.724194    2581 generic.go:356] "Generic (PLEG): container finished" podID="712ed70b-7ff9-4dfa-bd65-985ccf82f430" containerID="2ba6dc25540e91751625e0be21065c1bf77b276b99cda1e5e639c0d55cad0d81" exitCode=0
Sep 11 05:03:40.724540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:40.724242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr" event={"ID":"712ed70b-7ff9-4dfa-bd65-985ccf82f430","Type":"ContainerDied","Data":"2ba6dc25540e91751625e0be21065c1bf77b276b99cda1e5e639c0d55cad0d81"}
Sep 11 05:03:40.724540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:40.724301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr" event={"ID":"712ed70b-7ff9-4dfa-bd65-985ccf82f430","Type":"ContainerStarted","Data":"0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04"}
Sep 11 05:03:41.854661 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.854636    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:41.876936 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.876914    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:41.877054 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.876958    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:41.889533 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.889511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:41.923985 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.923960    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca\") pod \"712ed70b-7ff9-4dfa-bd65-985ccf82f430\" (UID: \"712ed70b-7ff9-4dfa-bd65-985ccf82f430\") "
Sep 11 05:03:41.924245 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:41.924225    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "712ed70b-7ff9-4dfa-bd65-985ccf82f430" (UID: "712ed70b-7ff9-4dfa-bd65-985ccf82f430"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:03:42.025225 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:42.025168    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/712ed70b-7ff9-4dfa-bd65-985ccf82f430-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:03:42.736817 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:42.736786    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"
Sep 11 05:03:42.736990 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:42.736784    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr" event={"ID":"712ed70b-7ff9-4dfa-bd65-985ccf82f430","Type":"ContainerDied","Data":"0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04"}
Sep 11 05:03:42.736990 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:42.736929    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c56cc819a901ce385abe757797a3a1059f0b277f31a39e17c640d07e0f79c04"
Sep 11 05:03:42.748845 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:42.748824    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:03:55.311880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:55.311841    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"]
Sep 11 05:03:55.314368 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:55.314341    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c1509-k7vfr"]
Sep 11 05:03:56.192301 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:03:56.192258    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="712ed70b-7ff9-4dfa-bd65-985ccf82f430" path="/var/lib/kubelet/pods/712ed70b-7ff9-4dfa-bd65-985ccf82f430/volumes"
Sep 11 05:04:01.924942 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:01.924902    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:04:01.925418 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:01.925121    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy" containerID="cri-o://11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890" gracePeriod=30
Sep 11 05:04:02.266606 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.266561    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"]
Sep 11 05:04:02.266923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.266910    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="712ed70b-7ff9-4dfa-bd65-985ccf82f430" containerName="revoke-keys"
Sep 11 05:04:02.266970 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.266924    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="712ed70b-7ff9-4dfa-bd65-985ccf82f430" containerName="revoke-keys"
Sep 11 05:04:02.267014 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.266998    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="712ed70b-7ff9-4dfa-bd65-985ccf82f430" containerName="revoke-keys"
Sep 11 05:04:02.270286 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.270269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:02.272401 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.272381    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:04:02.272560 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.272543    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:04:02.276626 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.276602    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"]
Sep 11 05:04:02.301095 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.301073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm\" (UID: \"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:02.402174 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.402144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm\" (UID: \"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:02.402729 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.402714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm\" (UID: \"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:02.582366 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.582278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:02.854530 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:02.854507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"]
Sep 11 05:04:02.855873 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:04:02.855847    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5239ba3d_7dbe_4ac0_b2ae_a59ec14161db.slice/crio-2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233 WatchSource:0}: Error finding container 2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233: Status 404 returned error can't find the container with id 2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233
Sep 11 05:04:03.860200 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:03.860112    2581 generic.go:356] "Generic (PLEG): container finished" podID="5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" containerID="4a14e9763a3a1b749487a0ca9539c8b482c090a4104249b445867a33220232de" exitCode=0
Sep 11 05:04:03.860200 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:03.860170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm" event={"ID":"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db","Type":"ContainerDied","Data":"4a14e9763a3a1b749487a0ca9539c8b482c090a4104249b445867a33220232de"}
Sep 11 05:04:03.860601 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:03.860222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm" event={"ID":"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db","Type":"ContainerStarted","Data":"2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233"}
Sep 11 05:04:04.986895 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:04.986870    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:05.025253 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.025227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca\") pod \"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db\" (UID: \"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db\") "
Sep 11 05:04:05.025637 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.025570    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" (UID: "5239ba3d-7dbe-4ac0-b2ae-a59ec14161db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:05.126923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.126837    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:05.872847 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.872819    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"
Sep 11 05:04:05.872847 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.872827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm" event={"ID":"5239ba3d-7dbe-4ac0-b2ae-a59ec14161db","Type":"ContainerDied","Data":"2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233"}
Sep 11 05:04:05.873048 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:05.872872    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fa29a78cd3a0cc82f646c047ca9bc8b31bbf324623cdd7c4dc04e7bdea20233"
Sep 11 05:04:06.444633 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:06.444606    2581 scope.go:117] "RemoveContainer" containerID="1a48ee3e1ac8ad9dc396da0b69a7bf5f01c82e5589f4bcbdcdcc55d323fd9257"
Sep 11 05:04:07.170144 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.170121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:04:07.245850 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.245820    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246018 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.245865    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246018 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.245885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246018 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.245910    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246018 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.245980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246058    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zkql\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246107    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246116    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert\") pod \"0519827c-f2ef-4f68-920d-de13080e1a25\" (UID: \"0519827c-f2ef-4f68-920d-de13080e1a25\") "
Sep 11 05:04:07.246229 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246188    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:07.246486 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data" (OuterVolumeSpecName: "istio-data") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:07.246486 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:07.246486 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246426    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:07.246739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246552    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.246739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246572    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0519827c-f2ef-4f68-920d-de13080e1a25-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.246739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246611    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.246739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246627    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.246739 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.246643    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.248199 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.248175    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:04:07.248326 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.248300    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token" (OuterVolumeSpecName: "istio-token") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:07.248405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.248383    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql" (OuterVolumeSpecName: "kube-api-access-5zkql") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "kube-api-access-5zkql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:07.248458 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.248438    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0519827c-f2ef-4f68-920d-de13080e1a25" (UID: "0519827c-f2ef-4f68-920d-de13080e1a25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:07.348082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.348044    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0519827c-f2ef-4f68-920d-de13080e1a25-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.348082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.348079    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0519827c-f2ef-4f68-920d-de13080e1a25-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.348082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.348088    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.348082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.348096    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zkql\" (UniqueName: \"kubernetes.io/projected/0519827c-f2ef-4f68-920d-de13080e1a25-kube-api-access-5zkql\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:07.885591 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.885553    2581 generic.go:356] "Generic (PLEG): container finished" podID="0519827c-f2ef-4f68-920d-de13080e1a25" containerID="11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890" exitCode=0
Sep 11 05:04:07.885973 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.885630    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"
Sep 11 05:04:07.885973 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.885643    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerDied","Data":"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"}
Sep 11 05:04:07.885973 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.885705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5" event={"ID":"0519827c-f2ef-4f68-920d-de13080e1a25","Type":"ContainerDied","Data":"91881e8a5cf69acafb672df30bd19627e7f315c67e0582463c4fe90ce871320a"}
Sep 11 05:04:07.885973 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.885728    2581 scope.go:117] "RemoveContainer" containerID="11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"
Sep 11 05:04:07.895956 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.895941    2581 scope.go:117] "RemoveContainer" containerID="4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"
Sep 11 05:04:07.903869 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.903852    2581 scope.go:117] "RemoveContainer" containerID="11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"
Sep 11 05:04:07.904122 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:07.904103    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890\": container with ID starting with 11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890 not found: ID does not exist" containerID="11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"
Sep 11 05:04:07.904190 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.904133    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890"} err="failed to get container status \"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890\": rpc error: code = NotFound desc = could not find container \"11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890\": container with ID starting with 11f85955899f9c245e8fde3dcb647719f7cbcc588f53d03bc4d67e8287664890 not found: ID does not exist"
Sep 11 05:04:07.904190 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.904159    2581 scope.go:117] "RemoveContainer" containerID="4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"
Sep 11 05:04:07.904428 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:07.904411    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b\": container with ID starting with 4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b not found: ID does not exist" containerID="4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"
Sep 11 05:04:07.904465 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.904434    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b"} err="failed to get container status \"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b\": rpc error: code = NotFound desc = could not find container \"4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b\": container with ID starting with 4780db69f660e6f4da0a5954815724f258a20ea39e78317c32f2186afe47129b not found: ID does not exist"
Sep 11 05:04:07.909810 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.909787    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:04:07.915883 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:07.915864    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9c1509-openshift-default-99bcb6bd8-sn2n5"]
Sep 11 05:04:08.197387 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:08.193570    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" path="/var/lib/kubelet/pods/0519827c-f2ef-4f68-920d-de13080e1a25/volumes"
Sep 11 05:04:12.940918 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:12.940884    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"]
Sep 11 05:04:12.943236 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:12.943215    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f5e6f7-796mm"]
Sep 11 05:04:14.191879 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:14.191851    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" path="/var/lib/kubelet/pods/5239ba3d-7dbe-4ac0-b2ae-a59ec14161db/volumes"
Sep 11 05:04:19.115762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.115724    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 05:04:19.116196 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.115979    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy" containerID="cri-o://b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561" gracePeriod=30
Sep 11 05:04:19.462500 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.462462    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"]
Sep 11 05:04:19.462908 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.462890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:19.462997 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.462910    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:19.462997 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.462946    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" containerName="revoke-keys"
Sep 11 05:04:19.462997 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.462955    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" containerName="revoke-keys"
Sep 11 05:04:19.463155 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.463049    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:19.463155 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.463061    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5239ba3d-7dbe-4ac0-b2ae-a59ec14161db" containerName="revoke-keys"
Sep 11 05:04:19.463155 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.463071    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:19.467471 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.467451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:19.470359 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.470144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:04:19.470522 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.470503    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:04:19.472278 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.472255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"]
Sep 11 05:04:19.562950 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.562919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt\" (UID: \"e6e74a72-3c35-4c15-aebc-aaedd0ba318b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:19.664000 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.663970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt\" (UID: \"e6e74a72-3c35-4c15-aebc-aaedd0ba318b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:19.664631 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.664614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt\" (UID: \"e6e74a72-3c35-4c15-aebc-aaedd0ba318b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:19.779763 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:19.779675    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:20.051622 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:20.051593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"]
Sep 11 05:04:20.052966 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:04:20.052940    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e74a72_3c35_4c15_aebc_aaedd0ba318b.slice/crio-39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692 WatchSource:0}: Error finding container 39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692: Status 404 returned error can't find the container with id 39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692
Sep 11 05:04:20.963004 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:20.962969    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6e74a72-3c35-4c15-aebc-aaedd0ba318b" containerID="09efbcb07c48aab77197bb38b91b998923c57384d50bf88a5fad109750b271c9" exitCode=0
Sep 11 05:04:20.963672 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:20.963041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt" event={"ID":"e6e74a72-3c35-4c15-aebc-aaedd0ba318b","Type":"ContainerDied","Data":"09efbcb07c48aab77197bb38b91b998923c57384d50bf88a5fad109750b271c9"}
Sep 11 05:04:20.963672 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:20.963113    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt" event={"ID":"e6e74a72-3c35-4c15-aebc-aaedd0ba318b","Type":"ContainerStarted","Data":"39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692"}
Sep 11 05:04:20.963808 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:20.963765    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:04:21.839128 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:21.839091    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:04:22.101090 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.101036    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:22.188884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.188851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca\") pod \"e6e74a72-3c35-4c15-aebc-aaedd0ba318b\" (UID: \"e6e74a72-3c35-4c15-aebc-aaedd0ba318b\") "
Sep 11 05:04:22.189191 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.189166    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6e74a72-3c35-4c15-aebc-aaedd0ba318b" (UID: "e6e74a72-3c35-4c15-aebc-aaedd0ba318b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:22.289730 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.289694    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e74a72-3c35-4c15-aebc-aaedd0ba318b-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:22.975919 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.975895    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"
Sep 11 05:04:22.975919 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.975897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt" event={"ID":"e6e74a72-3c35-4c15-aebc-aaedd0ba318b","Type":"ContainerDied","Data":"39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692"}
Sep 11 05:04:22.976140 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:22.975956    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39e0c25e9278fac65428f5f2e85806785f93fd06f526c62216775acd72031692"
Sep 11 05:04:24.362641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.362618    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 05:04:24.513056 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513020    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513252 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513099    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513252 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513160    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513408 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513468 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513529 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513473    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4spm\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513529 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513510    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513701 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.513701 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513556    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:24.513701 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket\") pod \"23ec5a88-1186-4a50-b54b-a611c22eba18\" (UID: \"23ec5a88-1186-4a50-b54b-a611c22eba18\") "
Sep 11 05:04:24.514105 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513866    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data" (OuterVolumeSpecName: "istio-data") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:24.514105 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.513935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:24.514105 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.514026    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23ec5a88-1186-4a50-b54b-a611c22eba18-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.514105 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.514045    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.514105 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.514060    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.514634 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.514119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:24.514634 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.514022    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:24.515812 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.515768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:04:24.516073 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.516051    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm" (OuterVolumeSpecName: "kube-api-access-z4spm") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "kube-api-access-z4spm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:24.516189 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.516171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:24.516237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.516210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token" (OuterVolumeSpecName: "istio-token") pod "23ec5a88-1186-4a50-b54b-a611c22eba18" (UID: "23ec5a88-1186-4a50-b54b-a611c22eba18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:24.615263 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615215    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.615263 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615258    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.615263 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615270    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.615263 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615279    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4spm\" (UniqueName: \"kubernetes.io/projected/23ec5a88-1186-4a50-b54b-a611c22eba18-kube-api-access-z4spm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.615629 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615288    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.615629 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.615296    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23ec5a88-1186-4a50-b54b-a611c22eba18-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:24.989883 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.989852    2581 generic.go:356] "Generic (PLEG): container finished" podID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerID="b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561" exitCode=0
Sep 11 05:04:24.990035 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.989900    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"
Sep 11 05:04:24.990035 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.989940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerDied","Data":"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"}
Sep 11 05:04:24.990035 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.989996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk" event={"ID":"23ec5a88-1186-4a50-b54b-a611c22eba18","Type":"ContainerDied","Data":"96e73edc309ca81fdfebe67afab3b7b211a14f771ef459473b4ebf41bfbaf524"}
Sep 11 05:04:24.990035 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.990019    2581 scope.go:117] "RemoveContainer" containerID="b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"
Sep 11 05:04:24.999869 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:24.999853    2581 scope.go:117] "RemoveContainer" containerID="2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"
Sep 11 05:04:25.009395 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.009373    2581 scope.go:117] "RemoveContainer" containerID="b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"
Sep 11 05:04:25.009704 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:25.009683    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561\": container with ID starting with b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561 not found: ID does not exist" containerID="b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"
Sep 11 05:04:25.009801 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.009712    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561"} err="failed to get container status \"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561\": rpc error: code = NotFound desc = could not find container \"b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561\": container with ID starting with b7b91ded0cd03329acf8df60983465c356cd4384be707d418d1903892fc70561 not found: ID does not exist"
Sep 11 05:04:25.009801 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.009728    2581 scope.go:117] "RemoveContainer" containerID="2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"
Sep 11 05:04:25.009967 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:25.009948    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea\": container with ID starting with 2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea not found: ID does not exist" containerID="2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"
Sep 11 05:04:25.010026 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.009976    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea"} err="failed to get container status \"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea\": rpc error: code = NotFound desc = could not find container \"2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea\": container with ID starting with 2aaad8463316513a1d2449d5eb979b4830707e596c2f5c52ed7f67748b855aea not found: ID does not exist"
Sep 11 05:04:25.011785 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.011761    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 05:04:25.014103 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:25.014084    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-f5e6f7-openshift-default-8c7d757b8-pcjhk"]
Sep 11 05:04:26.192688 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:26.192655    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" path="/var/lib/kubelet/pods/23ec5a88-1186-4a50-b54b-a611c22eba18/volumes"
Sep 11 05:04:30.111997 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:30.111964    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"]
Sep 11 05:04:30.114042 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:30.114023    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f81bf2-29818380-kxtsx"]
Sep 11 05:04:30.193130 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:30.193101    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7909a84a-4552-4016-8a73-ea0299df6383" path="/var/lib/kubelet/pods/7909a84a-4552-4016-8a73-ea0299df6383/volumes"
Sep 11 05:04:30.277339 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:30.277304    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"]
Sep 11 05:04:30.279336 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:30.279318    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f81bf2-rmpjt"]
Sep 11 05:04:32.192996 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:32.192957    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6e74a72-3c35-4c15-aebc-aaedd0ba318b" path="/var/lib/kubelet/pods/e6e74a72-3c35-4c15-aebc-aaedd0ba318b/volumes"
Sep 11 05:04:35.983260 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:35.983223    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" podUID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" containerName="main" containerID="cri-o://f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66" gracePeriod=45
Sep 11 05:04:36.222978 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.222952    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:04:36.330134 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330045    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330134 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330134 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330133    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330355 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330355 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czbkv\" (UniqueName: \"kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330355 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330325    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir\") pod \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\" (UID: \"258d5c5d-ad68-44fa-be11-ed289e92c6d6\") "
Sep 11 05:04:36.330490 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330317    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home" (OuterVolumeSpecName: "home") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:36.330490 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330403    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:36.330624 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330557    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:36.330718 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330700    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.330778 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330725    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.330778 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.330740    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.332328 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.332309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm" (OuterVolumeSpecName: "dshm") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:36.332413 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.332385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:36.332526 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.332507    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv" (OuterVolumeSpecName: "kube-api-access-czbkv") pod "258d5c5d-ad68-44fa-be11-ed289e92c6d6" (UID: "258d5c5d-ad68-44fa-be11-ed289e92c6d6"). InnerVolumeSpecName "kube-api-access-czbkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:36.431872 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.431833    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czbkv\" (UniqueName: \"kubernetes.io/projected/258d5c5d-ad68-44fa-be11-ed289e92c6d6-kube-api-access-czbkv\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.431872 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.431864    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258d5c5d-ad68-44fa-be11-ed289e92c6d6-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.431872 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.431873    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258d5c5d-ad68-44fa-be11-ed289e92c6d6-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:36.796864 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.796834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 05:04:36.797143 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.797101    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" containerID="cri-o://88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc" gracePeriod=30
Sep 11 05:04:36.859699 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:36.859643    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" podUID="52055a68-d0b7-426b-b193-6971ae7c34ca" containerName="main" containerID="cri-o://2cdbf1393aab91574c54cd2bd403d14a311ac6f8b83f52033275433460e41c1b" gracePeriod=45
Sep 11 05:04:37.060082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.060005    2581 generic.go:356] "Generic (PLEG): container finished" podID="52055a68-d0b7-426b-b193-6971ae7c34ca" containerID="2cdbf1393aab91574c54cd2bd403d14a311ac6f8b83f52033275433460e41c1b" exitCode=0
Sep 11 05:04:37.060448 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.060078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" event={"ID":"52055a68-d0b7-426b-b193-6971ae7c34ca","Type":"ContainerDied","Data":"2cdbf1393aab91574c54cd2bd403d14a311ac6f8b83f52033275433460e41c1b"}
Sep 11 05:04:37.061613 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.061568    2581 generic.go:356] "Generic (PLEG): container finished" podID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" containerID="f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66" exitCode=0
Sep 11 05:04:37.061728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.061620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" event={"ID":"258d5c5d-ad68-44fa-be11-ed289e92c6d6","Type":"ContainerDied","Data":"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"}
Sep 11 05:04:37.061728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.061665    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"
Sep 11 05:04:37.061728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.061678    2581 scope.go:117] "RemoveContainer" containerID="f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"
Sep 11 05:04:37.061871 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.061665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b" event={"ID":"258d5c5d-ad68-44fa-be11-ed289e92c6d6","Type":"ContainerDied","Data":"d32d4ed860862da9309ca3151d6fe75928b9441f59871140bfb7a25a7f86d9db"}
Sep 11 05:04:37.071323 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.071308    2581 scope.go:117] "RemoveContainer" containerID="f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"
Sep 11 05:04:37.071595 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:37.071555    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66\": container with ID starting with f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66 not found: ID does not exist" containerID="f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"
Sep 11 05:04:37.071703 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.071610    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66"} err="failed to get container status \"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66\": rpc error: code = NotFound desc = could not find container \"f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66\": container with ID starting with f3f8e61184518d365bcc56467b4952a51fc5fbb1bf3217134c6a28810ce97f66 not found: ID does not exist"
Sep 11 05:04:37.082995 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.082963    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:04:37.085170 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.085148    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c1858a/rate-test-model-c1858a-kserve-645bd9859f-xgw4b"]
Sep 11 05:04:37.115537 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.115518    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:04:37.240086 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240053    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240095    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240166    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scghr\" (UniqueName: \"kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240271 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs\") pod \"52055a68-d0b7-426b-b193-6971ae7c34ca\" (UID: \"52055a68-d0b7-426b-b193-6971ae7c34ca\") "
Sep 11 05:04:37.240528 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home" (OuterVolumeSpecName: "home") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:37.240528 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:37.240528 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.240444    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:37.242409 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.242385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm" (OuterVolumeSpecName: "dshm") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:37.242409 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.242389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:37.242409 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.242398    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr" (OuterVolumeSpecName: "kube-api-access-scghr") pod "52055a68-d0b7-426b-b193-6971ae7c34ca" (UID: "52055a68-d0b7-426b-b193-6971ae7c34ca"). InnerVolumeSpecName "kube-api-access-scghr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341617    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341656    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341670    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scghr\" (UniqueName: \"kubernetes.io/projected/52055a68-d0b7-426b-b193-6971ae7c34ca-kube-api-access-scghr\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341682    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52055a68-d0b7-426b-b193-6971ae7c34ca-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341696    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:37.341712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:37.341707    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52055a68-d0b7-426b-b193-6971ae7c34ca-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:38.069737 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.069704    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"
Sep 11 05:04:38.070232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.069697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7" event={"ID":"52055a68-d0b7-426b-b193-6971ae7c34ca","Type":"ContainerDied","Data":"9ac86c16df17e5210b7fd3c7ba5dfcaef53f60ab2dba7879dd5ad132068b4e15"}
Sep 11 05:04:38.070232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.069844    2581 scope.go:117] "RemoveContainer" containerID="2cdbf1393aab91574c54cd2bd403d14a311ac6f8b83f52033275433460e41c1b"
Sep 11 05:04:38.092210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.092177    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:04:38.096077 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.096054    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-16b799/rate-test-model-16b799-kserve-6f6ff664c-skzh7"]
Sep 11 05:04:38.192183 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.192149    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" path="/var/lib/kubelet/pods/258d5c5d-ad68-44fa-be11-ed289e92c6d6/volumes"
Sep 11 05:04:38.192527 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:38.192515    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52055a68-d0b7-426b-b193-6971ae7c34ca" path="/var/lib/kubelet/pods/52055a68-d0b7-426b-b193-6971ae7c34ca/volumes"
Sep 11 05:04:41.366306 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:41.366276    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 05:04:41.366728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:41.366330    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 05:04:42.040005 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.039985    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 05:04:42.077923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.077847    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.077923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.077892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078149 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.077923    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078149 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.077997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078149 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078027    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078149 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078072    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078149 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:42.078405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078153    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078189    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:42.078405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078196    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn49b\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078294    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:42.078405 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078300    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket\") pod \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\" (UID: \"ae1979b7-fc13-41e1-b848-2ff667e4ea67\") "
Sep 11 05:04:42.078682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078560    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:42.078682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078602    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.078682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078621    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.078682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078635    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.078900 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.078735    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:42.080409 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.080377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b" (OuterVolumeSpecName: "kube-api-access-zn49b") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "kube-api-access-zn49b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:42.080539 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.080462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:42.080857 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.080831    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:04:42.080952 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.080836    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae1979b7-fc13-41e1-b848-2ff667e4ea67" (UID: "ae1979b7-fc13-41e1-b848-2ff667e4ea67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:04:42.092396 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.092368    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerID="88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc" exitCode=0
Sep 11 05:04:42.092502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.092409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerDied","Data":"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"}
Sep 11 05:04:42.092502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.092421    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"
Sep 11 05:04:42.092502 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.092496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j" event={"ID":"ae1979b7-fc13-41e1-b848-2ff667e4ea67","Type":"ContainerDied","Data":"b4464ccfd1e90c6041f07ebcf430bf77bcb9ddfdd59a517c6f3ae5a19dc8414c"}
Sep 11 05:04:42.092654 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.092528    2581 scope.go:117] "RemoveContainer" containerID="88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"
Sep 11 05:04:42.102619 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.102566    2581 scope.go:117] "RemoveContainer" containerID="020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"
Sep 11 05:04:42.111266 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.111245    2581 scope.go:117] "RemoveContainer" containerID="88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"
Sep 11 05:04:42.111535 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:42.111519    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc\": container with ID starting with 88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc not found: ID does not exist" containerID="88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"
Sep 11 05:04:42.111639 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.111543    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc"} err="failed to get container status \"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc\": rpc error: code = NotFound desc = could not find container \"88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc\": container with ID starting with 88aecfff1c7db8eff5ff9fa406d0e86713bc723fcac9b608f457883a871e9edc not found: ID does not exist"
Sep 11 05:04:42.111639 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.111560    2581 scope.go:117] "RemoveContainer" containerID="020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"
Sep 11 05:04:42.111865 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:04:42.111843    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1\": container with ID starting with 020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1 not found: ID does not exist" containerID="020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"
Sep 11 05:04:42.111930 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.111871    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1"} err="failed to get container status \"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1\": rpc error: code = NotFound desc = could not find container \"020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1\": container with ID starting with 020f14d95caf74374c409515567caddcc76b9949eb3edef943975bef091855b1 not found: ID does not exist"
Sep 11 05:04:42.113727 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.113705    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 05:04:42.118633 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.118614    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f81bf2-openshift-default-55f59b48bb-47x5j"]
Sep 11 05:04:42.179317 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179294    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.179317 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179313    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.179317 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179322    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.179483 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179330    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.179483 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179339    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn49b\" (UniqueName: \"kubernetes.io/projected/ae1979b7-fc13-41e1-b848-2ff667e4ea67-kube-api-access-zn49b\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.179483 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.179348    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae1979b7-fc13-41e1-b848-2ff667e4ea67-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:04:42.191687 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:42.191662    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" path="/var/lib/kubelet/pods/ae1979b7-fc13-41e1-b848-2ff667e4ea67/volumes"
Sep 11 05:04:51.303593 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303552    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303885    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303897    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303908    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303914    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303927    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52055a68-d0b7-426b-b193-6971ae7c34ca" containerName="main"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303933    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52055a68-d0b7-426b-b193-6971ae7c34ca" containerName="main"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303943    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:51.303960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303950    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0519827c-f2ef-4f68-920d-de13080e1a25" containerName="istio-proxy"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303966    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" containerName="main"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303975    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" containerName="main"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303982    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6e74a72-3c35-4c15-aebc-aaedd0ba318b" containerName="revoke-keys"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.303989    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6e74a72-3c35-4c15-aebc-aaedd0ba318b" containerName="revoke-keys"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304071    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="258d5c5d-ad68-44fa-be11-ed289e92c6d6" containerName="main"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304083    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52055a68-d0b7-426b-b193-6971ae7c34ca" containerName="main"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304092    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304100    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304110    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:04:51.304208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.304119    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6e74a72-3c35-4c15-aebc-aaedd0ba318b" containerName="revoke-keys"
Sep 11 05:04:51.308699 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.308674    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.310986 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.310954    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-c4902f-openshift-default-dockercfg-vfh58\""
Sep 11 05:04:51.320539 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.320517    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:04:51.359252 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359428 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359671 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hzm\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.359671 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.359452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.460898 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.460867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.460907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.460946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.460979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461087 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461414 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hzm\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461414 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.461525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.461466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.462072 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.462044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.462191 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.462088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.462237 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.462212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.463514 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.463496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.463597 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.463557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.469740 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.469702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hzm\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.470626 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.470609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token\") pod \"e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") " pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.619376 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.619290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:51.895476 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.895453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:04:51.897076 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:04:51.897047    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1607c5ad_b863_4a98_a0c6_843f4eee20c6.slice/crio-7444ddf4e37202c6cc2ef089925dfbde520a7ea87fe0bcad64a74b103956e345 WatchSource:0}: Error finding container 7444ddf4e37202c6cc2ef089925dfbde520a7ea87fe0bcad64a74b103956e345: Status 404 returned error can't find the container with id 7444ddf4e37202c6cc2ef089925dfbde520a7ea87fe0bcad64a74b103956e345
Sep 11 05:04:51.899476 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.899442    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:04:51.899551 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.899513    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:04:51.899620 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:51.899548    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:04:52.151201 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:52.151104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerStarted","Data":"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"}
Sep 11 05:04:52.151201 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:52.151176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerStarted","Data":"7444ddf4e37202c6cc2ef089925dfbde520a7ea87fe0bcad64a74b103956e345"}
Sep 11 05:04:52.170922 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:52.170877    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podStartSLOduration=1.170861151 podStartE2EDuration="1.170861151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:04:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:04:52.169267678 +0000 UTC m=+1486.460742119" watchObservedRunningTime="2026-09-11 05:04:52.170861151 +0000 UTC m=+1486.462335610"
Sep 11 05:04:52.619620 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:52.619563    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:52.624572 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:52.624551    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:53.156159 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:53.156127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:04:53.157083 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:04:53.157067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:05:06.274682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:06.274653    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:05:06.277014 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:06.276994    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:05:12.278312 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.278282    2581 generic.go:356] "Generic (PLEG): container finished" podID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerID="4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1" exitCode=0
Sep 11 05:05:12.278756 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.278360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerDied","Data":"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"}
Sep 11 05:05:12.278756 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.278679    2581 scope.go:117] "RemoveContainer" containerID="4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"
Sep 11 05:05:12.279682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.279660    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:12.279754 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.279729    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:12.279794 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:12.279759    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:13.286986 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:13.286953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerStarted","Data":"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"}
Sep 11 05:05:13.619820 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:13.619728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:05:13.619999 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:13.619976    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Sep 11 05:05:13.620044 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:13.620027    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Sep 11 05:05:14.619707 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:14.619670    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Sep 11 05:05:14.620078 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:14.619737    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Sep 11 05:05:15.620167 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:15.620136    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Sep 11 05:05:15.620561 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:15.620189    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Sep 11 05:05:16.620006 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:16.619973    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Sep 11 05:05:16.620188 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:16.620028    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Sep 11 05:05:17.620488 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:17.620450    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Sep 11 05:05:17.620999 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:17.620514    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Sep 11 05:05:18.623396 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:18.623366    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:05:18.623796 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:18.623634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:05:18.624219 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:18.624203    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:05:28.248851 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.248770    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:05:28.249204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.249103    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:05:28.249204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.249113    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:05:28.249204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.249123    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:05:28.249204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.249128    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23ec5a88-1186-4a50-b54b-a611c22eba18" containerName="istio-proxy"
Sep 11 05:05:28.249204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.249189    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1979b7-fc13-41e1-b848-2ff667e4ea67" containerName="istio-proxy"
Sep 11 05:05:28.251436 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.251422    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.254667 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.254635    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-909c76\"/\"kube-root-ca.crt\""
Sep 11 05:05:28.254667 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.254647    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-909c76\"/\"openshift-service-ca.crt\""
Sep 11 05:05:28.254879 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.254690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-909c76\"/\"ipp-route-909c76-kserve-self-signed-certs\""
Sep 11 05:05:28.254879 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.254635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-909c76\"/\"default-dockercfg-fz2tw\""
Sep 11 05:05:28.260680 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.260655    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:05:28.402272 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.402272 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjkx\" (UniqueName: \"kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.402475 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.402475 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.402475 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.402475 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.402471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504028 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.503928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504028 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.503982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504028 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504311 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504311 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjkx\" (UniqueName: \"kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504311 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.504486 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.505899 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.505899 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.504978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.507560 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.507506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.507861 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.507838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.515989 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.515961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjkx\" (UniqueName: \"kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx\") pod \"ipp-route-909c76-kserve-6b76558d8c-j7ptl\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") " pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.561987 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.561951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:28.832921 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:28.832890    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:05:28.834144 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:05:28.834120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod150ade0e_0946_4226_b397_2b91c67b2f04.slice/crio-c1ef446ebc4732c7738498bb0deea6c39905646ed33c3f904af2c2f738a5fec0 WatchSource:0}: Error finding container c1ef446ebc4732c7738498bb0deea6c39905646ed33c3f904af2c2f738a5fec0: Status 404 returned error can't find the container with id c1ef446ebc4732c7738498bb0deea6c39905646ed33c3f904af2c2f738a5fec0
Sep 11 05:05:29.374055 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:29.374022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" event={"ID":"150ade0e-0946-4226-b397-2b91c67b2f04","Type":"ContainerStarted","Data":"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"}
Sep 11 05:05:29.374055 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:29.374066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" event={"ID":"150ade0e-0946-4226-b397-2b91c67b2f04","Type":"ContainerStarted","Data":"c1ef446ebc4732c7738498bb0deea6c39905646ed33c3f904af2c2f738a5fec0"}
Sep 11 05:05:29.392783 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:29.392735    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" podStartSLOduration=1.39272108 podStartE2EDuration="1.39272108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:05:29.391755339 +0000 UTC m=+1523.683229792" watchObservedRunningTime="2026-09-11 05:05:29.39272108 +0000 UTC m=+1523.684195517"
Sep 11 05:05:38.563164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:38.563127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:38.563683 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:38.563193    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:38.575753 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:38.575728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:39.439055 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:39.439026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:05:55.219471 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.219439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:05:55.221778 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.221762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.223936 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.223916    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d69099-openshift-default-dockercfg-k84q4\""
Sep 11 05:05:55.232331 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.232308    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:05:55.341201 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341378 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341378 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341378 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341378 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9cht\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.341663 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.341534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9cht\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442360 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442360 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442360 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442735 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442735 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442855 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442855 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.442855 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.442828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.443282 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.443260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.444596 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.444564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.444830 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.444813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.449930 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.449907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.450369 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.450348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9cht\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht\") pod \"e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") " pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.533211 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.533122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:55.808038 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.808013    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:05:55.809851 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:05:55.809824    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fcf39f_f9f6_4e9e_b017_8c92a883c830.slice/crio-cffdaaff6a23666a19095f264154bccca5dad5c6faa8d60ad496b598e37822c6 WatchSource:0}: Error finding container cffdaaff6a23666a19095f264154bccca5dad5c6faa8d60ad496b598e37822c6: Status 404 returned error can't find the container with id cffdaaff6a23666a19095f264154bccca5dad5c6faa8d60ad496b598e37822c6
Sep 11 05:05:55.812085 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.812055    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:55.812179 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.812116    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:55.812179 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:55.812144    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:05:56.528468 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:56.528423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerStarted","Data":"cf255561b2a60e53ccd87b7c8ba2432bf2418797215174a05cfd34230a51c13d"}
Sep 11 05:05:56.528990 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:56.528489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerStarted","Data":"cffdaaff6a23666a19095f264154bccca5dad5c6faa8d60ad496b598e37822c6"}
Sep 11 05:05:56.534298 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:56.534280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:56.539528 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:56.539506    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:56.548015 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:56.547974    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podStartSLOduration=1.54795975 podStartE2EDuration="1.54795975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:05:56.545655557 +0000 UTC m=+1550.837129996" watchObservedRunningTime="2026-09-11 05:05:56.54795975 +0000 UTC m=+1550.839434188"
Sep 11 05:05:57.534246 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:57.534216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:05:57.535181 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:05:57.535164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:06.494887 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:06.494855    2581 scope.go:117] "RemoveContainer" containerID="b262441928f99e08535f2bf5ab480d6a631b198173906e02b50dc087cb3b58c5"
Sep 11 05:06:06.503823 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:06.503775    2581 scope.go:117] "RemoveContainer" containerID="9bf59e3955b4e8add1c81f9123b0db176aeb3b97ff32f42ac34d84d2167e6a6a"
Sep 11 05:06:17.659481 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.659451    2581 generic.go:356] "Generic (PLEG): container finished" podID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerID="cf255561b2a60e53ccd87b7c8ba2432bf2418797215174a05cfd34230a51c13d" exitCode=0
Sep 11 05:06:17.659948 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.659526    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerDied","Data":"cf255561b2a60e53ccd87b7c8ba2432bf2418797215174a05cfd34230a51c13d"}
Sep 11 05:06:17.659948 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.659909    2581 scope.go:117] "RemoveContainer" containerID="cf255561b2a60e53ccd87b7c8ba2432bf2418797215174a05cfd34230a51c13d"
Sep 11 05:06:17.661080 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.661050    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:06:17.661144 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.661122    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:06:17.661197 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:17.661153    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 05:06:18.667450 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:18.667422    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerStarted","Data":"02e2621d181a9fc04c167c926aaf848040d1544e55bdaf264842670c7440a4eb"}
Sep 11 05:06:19.533880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:19.533849    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 11 05:06:19.534158 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:19.534129    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 11 05:06:19.534336 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:19.534322    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:20.534060 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:20.534020    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 11 05:06:20.534555 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:20.534091    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 11 05:06:21.533554 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:21.533515    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 11 05:06:21.533784 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:21.533607    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 11 05:06:22.534379 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:22.534339    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 11 05:06:22.535017 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:22.534968    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 11 05:06:23.534339 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:23.534300    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 11 05:06:23.534524 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:23.534395    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 11 05:06:24.055352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.055318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"]
Sep 11 05:06:24.057662 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.057646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:24.059612 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.059574    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:06:24.059612 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.059606    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:06:24.067220 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.067199    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"]
Sep 11 05:06:24.111616 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.111567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492\" (UID: \"ee9872d9-b8e6-43fa-be4b-6e1045f2e195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:24.212896 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.212866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492\" (UID: \"ee9872d9-b8e6-43fa-be4b-6e1045f2e195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:24.213419 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.213403    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492\" (UID: \"ee9872d9-b8e6-43fa-be4b-6e1045f2e195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:24.369070 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.369012    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:24.537762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.537730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:24.538128 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.538089    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:24.538881 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.538863    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:24.637914 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.637887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"]
Sep 11 05:06:24.640033 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:06:24.640005    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9872d9_b8e6_43fa_be4b_6e1045f2e195.slice/crio-e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d WatchSource:0}: Error finding container e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d: Status 404 returned error can't find the container with id e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d
Sep 11 05:06:24.701518 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:24.701488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492" event={"ID":"ee9872d9-b8e6-43fa-be4b-6e1045f2e195","Type":"ContainerStarted","Data":"e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d"}
Sep 11 05:06:25.708555 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:25.708526    2581 generic.go:356] "Generic (PLEG): container finished" podID="ee9872d9-b8e6-43fa-be4b-6e1045f2e195" containerID="fc31e6a34cac6af4cad4f2b2ad80b1c4929a53979b7b12608f4fe718231f4558" exitCode=0
Sep 11 05:06:25.709053 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:25.708615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492" event={"ID":"ee9872d9-b8e6-43fa-be4b-6e1045f2e195","Type":"ContainerDied","Data":"fc31e6a34cac6af4cad4f2b2ad80b1c4929a53979b7b12608f4fe718231f4558"}
Sep 11 05:06:26.836759 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:26.836732    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:26.938202 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:26.938164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca\") pod \"ee9872d9-b8e6-43fa-be4b-6e1045f2e195\" (UID: \"ee9872d9-b8e6-43fa-be4b-6e1045f2e195\") "
Sep 11 05:06:26.938528 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:26.938504    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee9872d9-b8e6-43fa-be4b-6e1045f2e195" (UID: "ee9872d9-b8e6-43fa-be4b-6e1045f2e195"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:27.039623 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:27.039540    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee9872d9-b8e6-43fa-be4b-6e1045f2e195-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:27.719522 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:27.719493    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"
Sep 11 05:06:27.719749 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:27.719541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492" event={"ID":"ee9872d9-b8e6-43fa-be4b-6e1045f2e195","Type":"ContainerDied","Data":"e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d"}
Sep 11 05:06:27.719749 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:27.719611    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0b18631eac153a8de8a99f5d5755edd35470a9cbad42df7f3fd5183a4849d0d"
Sep 11 05:06:30.331343 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:30.331298    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"]
Sep 11 05:06:30.333631 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:30.333609    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d69099-nh492"]
Sep 11 05:06:32.193455 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:32.193423    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9872d9-b8e6-43fa-be4b-6e1045f2e195" path="/var/lib/kubelet/pods/ee9872d9-b8e6-43fa-be4b-6e1045f2e195/volumes"
Sep 11 05:06:36.567675 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:36.567645    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:06:36.568066 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:36.567904    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" containerID="cri-o://02e2621d181a9fc04c167c926aaf848040d1544e55bdaf264842670c7440a4eb" gracePeriod=30
Sep 11 05:06:39.540115 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:39.540078    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 05:06:39.540487 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:39.540139    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 05:06:40.008888 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.008853    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"]
Sep 11 05:06:40.009263 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.009249    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9872d9-b8e6-43fa-be4b-6e1045f2e195" containerName="revoke-keys"
Sep 11 05:06:40.009316 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.009265    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9872d9-b8e6-43fa-be4b-6e1045f2e195" containerName="revoke-keys"
Sep 11 05:06:40.009352 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.009330    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9872d9-b8e6-43fa-be4b-6e1045f2e195" containerName="revoke-keys"
Sep 11 05:06:40.011205 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.011190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:40.013491 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.013469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:06:40.013659 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.013516    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:06:40.019637 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.019182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"]
Sep 11 05:06:40.151140 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.151106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw\" (UID: \"35a01045-7fd5-4a81-b6ad-d1be052e255d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:40.251947 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.251910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw\" (UID: \"35a01045-7fd5-4a81-b6ad-d1be052e255d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:40.252523 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.252504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw\" (UID: \"35a01045-7fd5-4a81-b6ad-d1be052e255d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:40.323933 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.323868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:40.592816 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.592793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"]
Sep 11 05:06:40.594439 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:06:40.594411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a01045_7fd5_4a81_b6ad_d1be052e255d.slice/crio-be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7 WatchSource:0}: Error finding container be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7: Status 404 returned error can't find the container with id be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7
Sep 11 05:06:40.793533 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:40.793506    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw" event={"ID":"35a01045-7fd5-4a81-b6ad-d1be052e255d","Type":"ContainerStarted","Data":"be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7"}
Sep 11 05:06:41.801924 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.801848    2581 generic.go:356] "Generic (PLEG): container finished" podID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerID="02e2621d181a9fc04c167c926aaf848040d1544e55bdaf264842670c7440a4eb" exitCode=0
Sep 11 05:06:41.801924 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.801912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerDied","Data":"02e2621d181a9fc04c167c926aaf848040d1544e55bdaf264842670c7440a4eb"}
Sep 11 05:06:41.802300 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.801967    2581 scope.go:117] "RemoveContainer" containerID="cf255561b2a60e53ccd87b7c8ba2432bf2418797215174a05cfd34230a51c13d"
Sep 11 05:06:41.804208 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.804184    2581 generic.go:356] "Generic (PLEG): container finished" podID="35a01045-7fd5-4a81-b6ad-d1be052e255d" containerID="ef35aa3635ba295a9b0b267ff6d5810f838f0e5671cf4a7ccaba947805a35656" exitCode=0
Sep 11 05:06:41.804330 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.804269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw" event={"ID":"35a01045-7fd5-4a81-b6ad-d1be052e255d","Type":"ContainerDied","Data":"ef35aa3635ba295a9b0b267ff6d5810f838f0e5671cf4a7ccaba947805a35656"}
Sep 11 05:06:41.817286 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.817265    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:41.967254 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967231    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967391 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967391 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967311    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9cht\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967391 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967339    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967391 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967419    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967455    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967499    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket\") pod \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\" (UID: \"61fcf39f-f9f6-4e9e-b017-8c92a883c830\") "
Sep 11 05:06:41.967641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:06:41.967880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967775    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data" (OuterVolumeSpecName: "istio-data") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:06:41.967880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967796    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:41.967880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:41.968013 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967997    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:06:41.968140 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.967929    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:06:41.969670 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.969650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token" (OuterVolumeSpecName: "istio-token") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:06:41.969967 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.969948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:06:41.970138 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.970117    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht" (OuterVolumeSpecName: "kube-api-access-k9cht") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "kube-api-access-k9cht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:06:41.970138 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:41.970118    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61fcf39f-f9f6-4e9e-b017-8c92a883c830" (UID: "61fcf39f-f9f6-4e9e-b017-8c92a883c830"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:06:42.068934 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068913    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.068934 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068932    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068941    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068948    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068956    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068964    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9cht\" (UniqueName: \"kubernetes.io/projected/61fcf39f-f9f6-4e9e-b017-8c92a883c830-kube-api-access-k9cht\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068972    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.069065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.068980    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fcf39f-f9f6-4e9e-b017-8c92a883c830-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:42.811288 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.811262    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"
Sep 11 05:06:42.811771 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.811296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq" event={"ID":"61fcf39f-f9f6-4e9e-b017-8c92a883c830","Type":"ContainerDied","Data":"cffdaaff6a23666a19095f264154bccca5dad5c6faa8d60ad496b598e37822c6"}
Sep 11 05:06:42.811771 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.811378    2581 scope.go:117] "RemoveContainer" containerID="02e2621d181a9fc04c167c926aaf848040d1544e55bdaf264842670c7440a4eb"
Sep 11 05:06:42.843239 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.843205    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:06:42.848085 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.848058    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d69099-openshift-default-64fb8b5994-ggjbq"]
Sep 11 05:06:42.936130 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:42.936112    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:43.074896 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.074835    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca\") pod \"35a01045-7fd5-4a81-b6ad-d1be052e255d\" (UID: \"35a01045-7fd5-4a81-b6ad-d1be052e255d\") "
Sep 11 05:06:43.075161 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.075141    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "35a01045-7fd5-4a81-b6ad-d1be052e255d" (UID: "35a01045-7fd5-4a81-b6ad-d1be052e255d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:43.175488 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.175463    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a01045-7fd5-4a81-b6ad-d1be052e255d-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:06:43.819410 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.819380    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"
Sep 11 05:06:43.819909 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.819416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw" event={"ID":"35a01045-7fd5-4a81-b6ad-d1be052e255d","Type":"ContainerDied","Data":"be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7"}
Sep 11 05:06:43.819909 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:43.819484    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be73ce04be24bdcc9c41007ac583cbac004736f6fa2f01607bb182f44fe64cb7"
Sep 11 05:06:44.193238 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:44.193148    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" path="/var/lib/kubelet/pods/61fcf39f-f9f6-4e9e-b017-8c92a883c830/volumes"
Sep 11 05:06:55.624523 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:55.624489    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"]
Sep 11 05:06:55.627919 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:55.627896    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-909c76-zmbnw"]
Sep 11 05:06:56.192497 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:06:56.192453    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a01045-7fd5-4a81-b6ad-d1be052e255d" path="/var/lib/kubelet/pods/35a01045-7fd5-4a81-b6ad-d1be052e255d/volumes"
Sep 11 05:07:00.962134 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:00.962096    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:07:06.529081 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:06.529051    2581 scope.go:117] "RemoveContainer" containerID="5ea4cf930d1e4d6bda0d122ff28b9ba3348ea928341bfa29add2248442d9cb4a"
Sep 11 05:07:15.981618 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:15.981531    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" podUID="150ade0e-0946-4226-b397-2b91c67b2f04" containerName="main" containerID="cri-o://b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579" gracePeriod=45
Sep 11 05:07:16.222880 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.222859    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:07:16.247519 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247519 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247511    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247723 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247723 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247723 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247723 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wjkx\" (UniqueName: \"kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx\") pod \"150ade0e-0946-4226-b397-2b91c67b2f04\" (UID: \"150ade0e-0946-4226-b397-2b91c67b2f04\") "
Sep 11 05:07:16.247937 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247775    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache" (OuterVolumeSpecName: "model-cache") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:07:16.247937 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home" (OuterVolumeSpecName: "home") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:07:16.247937 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.247924    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:07:16.248097 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.248052    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:16.248097 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.248069    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:16.248097 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.248083    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:16.250137 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.250110    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:07:16.250565 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.250539    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm" (OuterVolumeSpecName: "dshm") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:07:16.250565 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.250554    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx" (OuterVolumeSpecName: "kube-api-access-8wjkx") pod "150ade0e-0946-4226-b397-2b91c67b2f04" (UID: "150ade0e-0946-4226-b397-2b91c67b2f04"). InnerVolumeSpecName "kube-api-access-8wjkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:07:16.348550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.348514    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150ade0e-0946-4226-b397-2b91c67b2f04-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:16.348550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.348543    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150ade0e-0946-4226-b397-2b91c67b2f04-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:16.348550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:16.348555    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wjkx\" (UniqueName: \"kubernetes.io/projected/150ade0e-0946-4226-b397-2b91c67b2f04-kube-api-access-8wjkx\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:17.017752 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.017723    2581 generic.go:356] "Generic (PLEG): container finished" podID="150ade0e-0946-4226-b397-2b91c67b2f04" containerID="b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579" exitCode=0
Sep 11 05:07:17.018165 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.017772    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"
Sep 11 05:07:17.018165 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.017811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" event={"ID":"150ade0e-0946-4226-b397-2b91c67b2f04","Type":"ContainerDied","Data":"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"}
Sep 11 05:07:17.018165 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.017858    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl" event={"ID":"150ade0e-0946-4226-b397-2b91c67b2f04","Type":"ContainerDied","Data":"c1ef446ebc4732c7738498bb0deea6c39905646ed33c3f904af2c2f738a5fec0"}
Sep 11 05:07:17.018165 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.017875    2581 scope.go:117] "RemoveContainer" containerID="b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"
Sep 11 05:07:17.027847 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.027833    2581 scope.go:117] "RemoveContainer" containerID="b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"
Sep 11 05:07:17.028113 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:07:17.028094    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579\": container with ID starting with b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579 not found: ID does not exist" containerID="b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"
Sep 11 05:07:17.028171 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.028120    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579"} err="failed to get container status \"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579\": rpc error: code = NotFound desc = could not find container \"b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579\": container with ID starting with b0601d38f96c4c62fcbcbec54668028a13ce3d1266cac49b67b97632526a4579 not found: ID does not exist"
Sep 11 05:07:17.038713 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.038692    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:07:17.040728 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:17.040702    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-909c76/ipp-route-909c76-kserve-6b76558d8c-j7ptl"]
Sep 11 05:07:18.192251 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:18.192219    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="150ade0e-0946-4226-b397-2b91c67b2f04" path="/var/lib/kubelet/pods/150ade0e-0946-4226-b397-2b91c67b2f04/volumes"
Sep 11 05:07:43.881684 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.881651    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"]
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.881998    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150ade0e-0946-4226-b397-2b91c67b2f04" containerName="main"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882009    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="150ade0e-0946-4226-b397-2b91c67b2f04" containerName="main"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882022    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882028    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882044    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a01045-7fd5-4a81-b6ad-d1be052e255d" containerName="revoke-keys"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882052    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a01045-7fd5-4a81-b6ad-d1be052e255d" containerName="revoke-keys"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882113    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a01045-7fd5-4a81-b6ad-d1be052e255d" containerName="revoke-keys"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882123    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882131    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="150ade0e-0946-4226-b397-2b91c67b2f04" containerName="main"
Sep 11 05:07:43.882164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.882137    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:07:43.884888 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.884871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:43.886929 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.886908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:07:43.887017 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.886912    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:07:43.891793 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:43.891769    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"]
Sep 11 05:07:44.004473 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:44.004442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf\" (UID: \"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:44.105246 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:44.105211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf\" (UID: \"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:44.105870 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:44.105849    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf\" (UID: \"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:44.197430 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:44.197408    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:44.466075 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:44.466046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"]
Sep 11 05:07:44.467721 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:07:44.467692    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf5f543_0e68_4e3c_abbc_e79876e5f5b8.slice/crio-b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb WatchSource:0}: Error finding container b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb: Status 404 returned error can't find the container with id b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb
Sep 11 05:07:45.172434 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:45.172398    2581 generic.go:356] "Generic (PLEG): container finished" podID="5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" containerID="e086cd1cd9dd46f5b8f279b95cf8c47181d027e09db9c14a796d99d4106f1c82" exitCode=0
Sep 11 05:07:45.172877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:45.172456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf" event={"ID":"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8","Type":"ContainerDied","Data":"e086cd1cd9dd46f5b8f279b95cf8c47181d027e09db9c14a796d99d4106f1c82"}
Sep 11 05:07:45.172877 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:45.172507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf" event={"ID":"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8","Type":"ContainerStarted","Data":"b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb"}
Sep 11 05:07:46.302001 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:46.301981    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:46.324550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:46.324523    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca\") pod \"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8\" (UID: \"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8\") "
Sep 11 05:07:46.324867 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:46.324847    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" (UID: "5cf5f543-0e68-4e3c-abbc-e79876e5f5b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:07:46.425554 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:46.425517    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:07:47.185532 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:47.185498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf" event={"ID":"5cf5f543-0e68-4e3c-abbc-e79876e5f5b8","Type":"ContainerDied","Data":"b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb"}
Sep 11 05:07:47.185532 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:47.185519    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"
Sep 11 05:07:47.185775 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:47.185547    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4656c065f3ede162cd5ceaabd15f3b66bca884a0219d76e757a9ffb32c9c7fb"
Sep 11 05:07:54.996906 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:54.996867    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"]
Sep 11 05:07:54.999027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:54.998995    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c4902f-vb7rf"]
Sep 11 05:07:56.198432 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:07:56.198401    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" path="/var/lib/kubelet/pods/5cf5f543-0e68-4e3c-abbc-e79876e5f5b8/volumes"
Sep 11 05:08:01.296172 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.296138    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:08:01.296602 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.296406    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" containerID="cri-o://af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a" gracePeriod=30
Sep 11 05:08:01.637656 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.637563    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"]
Sep 11 05:08:01.637955 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.637942    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" containerName="revoke-keys"
Sep 11 05:08:01.637998 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.637957    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" containerName="revoke-keys"
Sep 11 05:08:01.637998 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.637977    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:08:01.637998 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.637982    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="61fcf39f-f9f6-4e9e-b017-8c92a883c830" containerName="istio-proxy"
Sep 11 05:08:01.638093 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.638041    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cf5f543-0e68-4e3c-abbc-e79876e5f5b8" containerName="revoke-keys"
Sep 11 05:08:01.641164 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.641149    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:01.643418 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.643390    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:08:01.643563 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.643449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:08:01.647463 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.647437    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"]
Sep 11 05:08:01.760478 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.760446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq\" (UID: \"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:01.861384 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.861346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq\" (UID: \"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:01.862024 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.862006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq\" (UID: \"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:01.952473 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:01.952447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:02.221331 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:02.221307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"]
Sep 11 05:08:02.222848 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:08:02.222815    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd8c21c_a094_42ee_8d16_4a76f8bb3c7e.slice/crio-031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270 WatchSource:0}: Error finding container 031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270: Status 404 returned error can't find the container with id 031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270
Sep 11 05:08:02.267350 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:02.267327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq" event={"ID":"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e","Type":"ContainerStarted","Data":"031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270"}
Sep 11 05:08:03.275462 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:03.275433    2581 generic.go:356] "Generic (PLEG): container finished" podID="ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" containerID="2de4a93b6994b31dea171cdcbeacbdc2051c364fc7c618fa21d7710e0e7739ac" exitCode=0
Sep 11 05:08:03.275920 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:03.275515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq" event={"ID":"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e","Type":"ContainerDied","Data":"2de4a93b6994b31dea171cdcbeacbdc2051c364fc7c618fa21d7710e0e7739ac"}
Sep 11 05:08:03.625923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:03.625834    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 05:08:03.625923 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:03.625898    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 05:08:04.407484 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:04.407460    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:04.485287 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:04.485259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca\") pod \"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e\" (UID: \"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e\") "
Sep 11 05:08:04.485635 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:04.485614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" (UID: "ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:04.586947 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:04.586872    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:05.288265 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:05.288237    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"
Sep 11 05:08:05.288439 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:05.288283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq" event={"ID":"ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e","Type":"ContainerDied","Data":"031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270"}
Sep 11 05:08:05.288439 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:05.288329    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="031142cbd817a6e7f215b2bed739c9bd22f1117bffdb4b373db613e9a28c2270"
Sep 11 05:08:06.540767 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.540742    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:08:06.607145 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607116    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607145 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607171    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607206    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607247    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hzm\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607398 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket\") pod \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\" (UID: \"1607c5ad-b863-4a98-a0c6-843f4eee20c6\") "
Sep 11 05:08:06.607761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:06.607761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:06.607761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607641    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:06.607761 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607755    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.607969 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607769    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.607969 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:06.607969 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607894    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.607969 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.607895    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:06.609695 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.609671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:06.609820 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.609729    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm" (OuterVolumeSpecName: "kube-api-access-n2hzm") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "kube-api-access-n2hzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:06.609948 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.609932    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:08:06.610044 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.610028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1607c5ad-b863-4a98-a0c6-843f4eee20c6" (UID: "1607c5ad-b863-4a98-a0c6-843f4eee20c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:06.708762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708727    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.708762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708756    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.708762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708768    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hzm\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-kube-api-access-n2hzm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.709006 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708780    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.709006 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708791    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1607c5ad-b863-4a98-a0c6-843f4eee20c6-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:06.709006 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:06.708802    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1607c5ad-b863-4a98-a0c6-843f4eee20c6-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:07.299978 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.299949    2581 generic.go:356] "Generic (PLEG): container finished" podID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerID="af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a" exitCode=0
Sep 11 05:08:07.300197 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.299997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"
Sep 11 05:08:07.300197 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.300045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerDied","Data":"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"}
Sep 11 05:08:07.300197 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.300106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc" event={"ID":"1607c5ad-b863-4a98-a0c6-843f4eee20c6","Type":"ContainerDied","Data":"7444ddf4e37202c6cc2ef089925dfbde520a7ea87fe0bcad64a74b103956e345"}
Sep 11 05:08:07.300197 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.300137    2581 scope.go:117] "RemoveContainer" containerID="af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"
Sep 11 05:08:07.312789 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.312688    2581 scope.go:117] "RemoveContainer" containerID="4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"
Sep 11 05:08:07.321907 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.321887    2581 scope.go:117] "RemoveContainer" containerID="af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"
Sep 11 05:08:07.322212 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:08:07.322193    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a\": container with ID starting with af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a not found: ID does not exist" containerID="af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"
Sep 11 05:08:07.322256 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.322222    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a"} err="failed to get container status \"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a\": rpc error: code = NotFound desc = could not find container \"af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a\": container with ID starting with af2cf3ce2ef4b0046342cbff89f0d88be4116ef264c3b72d7d5c64992496151a not found: ID does not exist"
Sep 11 05:08:07.322256 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.322243    2581 scope.go:117] "RemoveContainer" containerID="4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"
Sep 11 05:08:07.322515 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:08:07.322501    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1\": container with ID starting with 4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1 not found: ID does not exist" containerID="4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"
Sep 11 05:08:07.322558 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.322519    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1"} err="failed to get container status \"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1\": rpc error: code = NotFound desc = could not find container \"4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1\": container with ID starting with 4532b24fdde9ee11969792fe9dde5566b6fb8e72eb7455e783bffc60742a20d1 not found: ID does not exist"
Sep 11 05:08:07.323910 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.323892    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:08:07.326061 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:07.326040    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-c4902f-openshift-default-69c7c9569b-m9bcc"]
Sep 11 05:08:08.192952 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:08.192923    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" path="/var/lib/kubelet/pods/1607c5ad-b863-4a98-a0c6-843f4eee20c6/volumes"
Sep 11 05:08:12.023765 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:12.023731    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"]
Sep 11 05:08:12.027147 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:12.027120    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-c1858a-29818380-p6lgm"]
Sep 11 05:08:12.192830 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:12.192787    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948d3c1b-4f1d-4c84-b88f-848ff466a88a" path="/var/lib/kubelet/pods/948d3c1b-4f1d-4c84-b88f-848ff466a88a/volumes"
Sep 11 05:08:12.193104 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:12.193088    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"]
Sep 11 05:08:12.194884 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:12.194864    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-c1858a-h7btq"]
Sep 11 05:08:14.192473 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:14.192437    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" path="/var/lib/kubelet/pods/ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e/volumes"
Sep 11 05:08:19.058094 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058056    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"]
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058361    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058371    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058383    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" containerName="revoke-keys"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058389    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" containerName="revoke-keys"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058408    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058414    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058465    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1607c5ad-b863-4a98-a0c6-843f4eee20c6" containerName="istio-proxy"
Sep 11 05:08:19.058540 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.058479    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd8c21c-a094-42ee-8d16-4a76f8bb3c7e" containerName="revoke-keys"
Sep 11 05:08:19.062895 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.062877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:19.065241 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.065212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 05:08:19.065401 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.065374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:08:19.067924 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.067903    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"]
Sep 11 05:08:19.218824 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.218788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl\" (UID: \"a17d7efb-ae34-4d0a-9f29-d8eb65277913\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:19.320254 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.320169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl\" (UID: \"a17d7efb-ae34-4d0a-9f29-d8eb65277913\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:19.320836 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.320818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl\" (UID: \"a17d7efb-ae34-4d0a-9f29-d8eb65277913\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:19.375437 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.375405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:19.646611 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:19.646564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"]
Sep 11 05:08:19.648658 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:08:19.648630    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17d7efb_ae34_4d0a_9f29_d8eb65277913.slice/crio-4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91 WatchSource:0}: Error finding container 4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91: Status 404 returned error can't find the container with id 4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91
Sep 11 05:08:20.371268 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:20.371233    2581 generic.go:356] "Generic (PLEG): container finished" podID="a17d7efb-ae34-4d0a-9f29-d8eb65277913" containerID="b39f7bc0f60f2f2234320380a01f75b69f0f721801c9a8bae54502f1af94702b" exitCode=0
Sep 11 05:08:20.371708 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:20.371324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl" event={"ID":"a17d7efb-ae34-4d0a-9f29-d8eb65277913","Type":"ContainerDied","Data":"b39f7bc0f60f2f2234320380a01f75b69f0f721801c9a8bae54502f1af94702b"}
Sep 11 05:08:20.371708 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:20.371372    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl" event={"ID":"a17d7efb-ae34-4d0a-9f29-d8eb65277913","Type":"ContainerStarted","Data":"4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91"}
Sep 11 05:08:21.503879 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:21.503857    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:21.640796 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:21.640694    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca\") pod \"a17d7efb-ae34-4d0a-9f29-d8eb65277913\" (UID: \"a17d7efb-ae34-4d0a-9f29-d8eb65277913\") "
Sep 11 05:08:21.641032 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:21.641011    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a17d7efb-ae34-4d0a-9f29-d8eb65277913" (UID: "a17d7efb-ae34-4d0a-9f29-d8eb65277913"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:21.741985 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:21.741944    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a17d7efb-ae34-4d0a-9f29-d8eb65277913-maas-api-service-ca\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:22.384244 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:22.384209    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"
Sep 11 05:08:22.384244 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:22.384225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl" event={"ID":"a17d7efb-ae34-4d0a-9f29-d8eb65277913","Type":"ContainerDied","Data":"4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91"}
Sep 11 05:08:22.384503 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:22.384275    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4867e2446992ee70953c588bd882769b8cd81711ec4b107f8b1e318a77901f91"
Sep 11 05:08:29.613687 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:29.613605    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"]
Sep 11 05:08:29.615545 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:29.615522    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16b799-r4xvl"]
Sep 11 05:08:30.192126 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:30.192093    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a17d7efb-ae34-4d0a-9f29-d8eb65277913" path="/var/lib/kubelet/pods/a17d7efb-ae34-4d0a-9f29-d8eb65277913/volumes"
Sep 11 05:08:36.063373 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:36.063336    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 05:08:36.063844 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:36.063600    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy" containerID="cri-o://13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766" gracePeriod=30
Sep 11 05:08:41.315448 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.315424    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 05:08:41.406027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.405947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.405978    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406005    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqdcp\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406045    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406267    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406283    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406321    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert\") pod \"4b009752-948a-4724-9a61-45af15c8b7f7\" (UID: \"4b009752-948a-4724-9a61-45af15c8b7f7\") "
Sep 11 05:08:41.406332 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406290    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406387    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406637    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-data\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406658    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406671    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-workload-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.406717 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406687    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-credential-socket\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.406956 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.406732    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:41.408480 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.408460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:41.408629 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.408553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 05:08:41.408866 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.408851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:08:41.408866 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.408853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp" (OuterVolumeSpecName: "kube-api-access-zqdcp") pod "4b009752-948a-4724-9a61-45af15c8b7f7" (UID: "4b009752-948a-4724-9a61-45af15c8b7f7"). InnerVolumeSpecName "kube-api-access-zqdcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:41.491433 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.491396    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b009752-948a-4724-9a61-45af15c8b7f7" containerID="13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766" exitCode=0
Sep 11 05:08:41.491641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.491443    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"
Sep 11 05:08:41.491641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.491484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerDied","Data":"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"}
Sep 11 05:08:41.491641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.491539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6" event={"ID":"4b009752-948a-4724-9a61-45af15c8b7f7","Type":"ContainerDied","Data":"a9e5efe4064f8b59c349345108771b6435c130a707b1f5fdfb13cace9e61e17a"}
Sep 11 05:08:41.491641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.491562    2581 scope.go:117] "RemoveContainer" containerID="13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"
Sep 11 05:08:41.501381 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.501365    2581 scope.go:117] "RemoveContainer" containerID="cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"
Sep 11 05:08:41.508031 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.508012    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b009752-948a-4724-9a61-45af15c8b7f7-istio-envoy\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.508031 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.508033    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b009752-948a-4724-9a61-45af15c8b7f7-istiod-ca-cert\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.508157 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.508043    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqdcp\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-kube-api-access-zqdcp\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.508157 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.508052    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b009752-948a-4724-9a61-45af15c8b7f7-istio-podinfo\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.508157 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.508061    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b009752-948a-4724-9a61-45af15c8b7f7-istio-token\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:08:41.509896 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.509847    2581 scope.go:117] "RemoveContainer" containerID="13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"
Sep 11 05:08:41.510444 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:08:41.510351    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766\": container with ID starting with 13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766 not found: ID does not exist" containerID="13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"
Sep 11 05:08:41.510444 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.510386    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766"} err="failed to get container status \"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766\": rpc error: code = NotFound desc = could not find container \"13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766\": container with ID starting with 13fdc2bb000a549e1bc14dbad6ade40ffae2cce225856c08a892cda5d2d8d766 not found: ID does not exist"
Sep 11 05:08:41.510444 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.510430    2581 scope.go:117] "RemoveContainer" containerID="cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"
Sep 11 05:08:41.510741 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:08:41.510726    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104\": container with ID starting with cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104 not found: ID does not exist" containerID="cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"
Sep 11 05:08:41.510810 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.510747    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104"} err="failed to get container status \"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104\": rpc error: code = NotFound desc = could not find container \"cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104\": container with ID starting with cdc50d8875b8e99aa9417c1cca9cf9e9c614a05a350f71887642beea81e4d104 not found: ID does not exist"
Sep 11 05:08:41.512550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.512530    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 05:08:41.517991 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:41.517972    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-16b799-openshift-default-796fccb654-wc5q6"]
Sep 11 05:08:42.192030 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:08:42.191999    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" path="/var/lib/kubelet/pods/4b009752-948a-4724-9a61-45af15c8b7f7/volumes"
Sep 11 05:09:06.569561 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:09:06.569533    2581 scope.go:117] "RemoveContainer" containerID="8a1a166b8f8ad40c270191011984f7887b7c4805ef969cc11c997bfaac323d11"
Sep 11 05:10:06.313319 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:10:06.313290    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:10:06.313817 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:10:06.313551    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:10:06.598459 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:10:06.598377    2581 scope.go:117] "RemoveContainer" containerID="2ba6dc25540e91751625e0be21065c1bf77b276b99cda1e5e639c0d55cad0d81"
Sep 11 05:10:06.607250 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:10:06.607229    2581 scope.go:117] "RemoveContainer" containerID="e554685f135c8c30ecd3bab3f3913c0f81e901292a2e57f0183e6f58d9d94a83"
Sep 11 05:10:06.615170 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:10:06.615148    2581 scope.go:117] "RemoveContainer" containerID="4a14e9763a3a1b749487a0ca9539c8b482c090a4104249b445867a33220232de"
Sep 11 05:11:06.641349 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:11:06.641313    2581 scope.go:117] "RemoveContainer" containerID="09efbcb07c48aab77197bb38b91b998923c57384d50bf88a5fad109750b271c9"
Sep 11 05:13:04.657463 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657429    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657747    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657758    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657769    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a17d7efb-ae34-4d0a-9f29-d8eb65277913" containerName="revoke-keys"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657774    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a17d7efb-ae34-4d0a-9f29-d8eb65277913" containerName="revoke-keys"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657791    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657796    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657848    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:04.657891 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.657857    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a17d7efb-ae34-4d0a-9f29-d8eb65277913" containerName="revoke-keys"
Sep 11 05:13:04.660819 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.660802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.662941 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.662911    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-60eadfb9-kserve-self-signed-certs\""
Sep 11 05:13:04.668525 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.668504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:13:04.771201 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.771375 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.771375 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.771375 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.771375 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw26z\" (UniqueName: \"kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.771543 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.771384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872480 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872480 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872815 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw26z\" (UniqueName: \"kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872815 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872815 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.872815 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.873056 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.872970    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.873056 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.873024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.873140 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.873056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.874864 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.874845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.875144 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.875128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.881372 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.881348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw26z\" (UniqueName: \"kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z\") pod \"e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") " pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:04.971154 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:04.971121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:05.243867 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:05.243840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:13:05.245190 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:13:05.245153    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13a06733_5871_4d87_8774_3d5d432af24d.slice/crio-fb3cd8c542ab1c090c8dc7ef8d106928967f73d3b502b33e393b9f9ed8db1f1c WatchSource:0}: Error finding container fb3cd8c542ab1c090c8dc7ef8d106928967f73d3b502b33e393b9f9ed8db1f1c: Status 404 returned error can't find the container with id fb3cd8c542ab1c090c8dc7ef8d106928967f73d3b502b33e393b9f9ed8db1f1c
Sep 11 05:13:05.867794 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:05.867758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" event={"ID":"13a06733-5871-4d87-8774-3d5d432af24d","Type":"ContainerStarted","Data":"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"}
Sep 11 05:13:05.868281 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:05.867804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" event={"ID":"13a06733-5871-4d87-8774-3d5d432af24d","Type":"ContainerStarted","Data":"fb3cd8c542ab1c090c8dc7ef8d106928967f73d3b502b33e393b9f9ed8db1f1c"}
Sep 11 05:13:05.883466 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:05.883420    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" podStartSLOduration=1.883406027 podStartE2EDuration="1.883406027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:13:05.881862293 +0000 UTC m=+1980.173336730" watchObservedRunningTime="2026-09-11 05:13:05.883406027 +0000 UTC m=+1980.174880463"
Sep 11 05:13:06.669854 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:06.669823    2581 scope.go:117] "RemoveContainer" containerID="fc31e6a34cac6af4cad4f2b2ad80b1c4929a53979b7b12608f4fe718231f4558"
Sep 11 05:13:06.678462 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:06.678444    2581 scope.go:117] "RemoveContainer" containerID="ef35aa3635ba295a9b0b267ff6d5810f838f0e5671cf4a7ccaba947805a35656"
Sep 11 05:13:14.971682 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:14.971646    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:14.972184 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:14.971801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:14.984487 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:14.984464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:15.927083 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:15.927057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:13:18.447541 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.447507    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:13:18.447910 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.447872    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b009752-948a-4724-9a61-45af15c8b7f7" containerName="istio-proxy"
Sep 11 05:13:18.450762 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.450748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.453003 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.452981    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-60eadfb9-kserve-self-signed-certs\""
Sep 11 05:13:18.461090 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.461067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:13:18.484354 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.484354 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.484535 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpft7\" (UniqueName: \"kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.484535 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.484535 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.484535 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.484515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585658 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585806 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585806 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585893 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585930 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.585968 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpft7\" (UniqueName: \"kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.586012 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.585986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.586096 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.586080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.586284 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.586263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.587970 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.587947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.588277 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.588259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.593914 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.593894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpft7\" (UniqueName: \"kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7\") pod \"e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") " pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:18.759563 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:18.759491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:19.032535 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:19.032510    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:13:19.034163 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:13:19.034137    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba00eb5e_6dee_4793_a3b9_02eb12afe784.slice/crio-3e9c8443e5a3aaa3aef5ebc5b2dc8349e87bdcd17e3675d657d8b17b41815250 WatchSource:0}: Error finding container 3e9c8443e5a3aaa3aef5ebc5b2dc8349e87bdcd17e3675d657d8b17b41815250: Status 404 returned error can't find the container with id 3e9c8443e5a3aaa3aef5ebc5b2dc8349e87bdcd17e3675d657d8b17b41815250
Sep 11 05:13:19.942493 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:19.942456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" event={"ID":"ba00eb5e-6dee-4793-a3b9-02eb12afe784","Type":"ContainerStarted","Data":"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"}
Sep 11 05:13:19.942493 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:19.942500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" event={"ID":"ba00eb5e-6dee-4793-a3b9-02eb12afe784","Type":"ContainerStarted","Data":"3e9c8443e5a3aaa3aef5ebc5b2dc8349e87bdcd17e3675d657d8b17b41815250"}
Sep 11 05:13:19.958448 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:19.958408    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" podStartSLOduration=1.9583949029999999 podStartE2EDuration="1.958394903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:13:19.957616183 +0000 UTC m=+1994.249090617" watchObservedRunningTime="2026-09-11 05:13:19.958394903 +0000 UTC m=+1994.249869340"
Sep 11 05:13:28.759960 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:28.759925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:28.760404 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:28.759970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:28.772076 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:28.772055    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:29.001740 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:29.001714    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:13:47.388550 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:47.388515    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:13:48.572123 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:13:48.572096    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:14:02.407685 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.407638    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" podUID="13a06733-5871-4d87-8774-3d5d432af24d" containerName="main" containerID="cri-o://c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36" gracePeriod=45
Sep 11 05:14:02.640188 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.640162    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:14:02.740936 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.740913    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741094 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.740949    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741094 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.740992    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741094 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741063    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741266 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741266 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741141    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw26z\" (UniqueName: \"kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z\") pod \"13a06733-5871-4d87-8774-3d5d432af24d\" (UID: \"13a06733-5871-4d87-8774-3d5d432af24d\") "
Sep 11 05:14:02.741266 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741187    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache" (OuterVolumeSpecName: "model-cache") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:02.741395 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home" (OuterVolumeSpecName: "home") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:02.741395 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741285    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:02.741461 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741417    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:02.741461 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741430    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:02.741461 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.741456    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:02.743462 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.743442    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm" (OuterVolumeSpecName: "dshm") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:02.743561 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.743462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z" (OuterVolumeSpecName: "kube-api-access-jw26z") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "kube-api-access-jw26z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:14:02.743561 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.743532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13a06733-5871-4d87-8774-3d5d432af24d" (UID: "13a06733-5871-4d87-8774-3d5d432af24d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:14:02.842883 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.842854    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13a06733-5871-4d87-8774-3d5d432af24d-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:02.842883 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.842880    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13a06733-5871-4d87-8774-3d5d432af24d-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:02.843126 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:02.842895    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw26z\" (UniqueName: \"kubernetes.io/projected/13a06733-5871-4d87-8774-3d5d432af24d-kube-api-access-jw26z\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:03.170571 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.170481    2581 generic.go:356] "Generic (PLEG): container finished" podID="13a06733-5871-4d87-8774-3d5d432af24d" containerID="c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36" exitCode=0
Sep 11 05:14:03.170571 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.170531    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"
Sep 11 05:14:03.170772 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.170574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" event={"ID":"13a06733-5871-4d87-8774-3d5d432af24d","Type":"ContainerDied","Data":"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"}
Sep 11 05:14:03.170772 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.170641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47" event={"ID":"13a06733-5871-4d87-8774-3d5d432af24d","Type":"ContainerDied","Data":"fb3cd8c542ab1c090c8dc7ef8d106928967f73d3b502b33e393b9f9ed8db1f1c"}
Sep 11 05:14:03.170772 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.170660    2581 scope.go:117] "RemoveContainer" containerID="c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"
Sep 11 05:14:03.180953 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.180938    2581 scope.go:117] "RemoveContainer" containerID="c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"
Sep 11 05:14:03.181223 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:14:03.181203    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36\": container with ID starting with c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36 not found: ID does not exist" containerID="c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"
Sep 11 05:14:03.181314 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.181229    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36"} err="failed to get container status \"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36\": rpc error: code = NotFound desc = could not find container \"c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36\": container with ID starting with c2df58f729bbb2027d4492bbe5e592f6586501d883b1921e008611ad749b1a36 not found: ID does not exist"
Sep 11 05:14:03.191630 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.191601    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:14:03.197615 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.197598    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-60eadfb9-kserve-5cd8495979-nfb47"]
Sep 11 05:14:03.590961 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.590906    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" podUID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" containerName="main" containerID="cri-o://5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5" gracePeriod=45
Sep 11 05:14:03.832665 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.832641    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:14:03.952019 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.951999    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952193 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952037    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpft7\" (UniqueName: \"kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952193 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952193 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952126    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952193 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952177    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952416 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952207    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm\") pod \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\" (UID: \"ba00eb5e-6dee-4793-a3b9-02eb12afe784\") "
Sep 11 05:14:03.952416 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home" (OuterVolumeSpecName: "home") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:03.952416 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:03.952416 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:03.952645 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952508    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tmp-dir\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:03.952645 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952526    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-home\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:03.952645 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.952540    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-model-cache\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:03.954322 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.954296    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm" (OuterVolumeSpecName: "dshm") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:14:03.954430 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.954341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:14:03.954430 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:03.954385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7" (OuterVolumeSpecName: "kube-api-access-zpft7") pod "ba00eb5e-6dee-4793-a3b9-02eb12afe784" (UID: "ba00eb5e-6dee-4793-a3b9-02eb12afe784"). InnerVolumeSpecName "kube-api-access-zpft7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:14:04.053801 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.053773    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba00eb5e-6dee-4793-a3b9-02eb12afe784-tls-certs\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:04.053801 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.053796    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba00eb5e-6dee-4793-a3b9-02eb12afe784-dshm\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:04.053988 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.053810    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpft7\" (UniqueName: \"kubernetes.io/projected/ba00eb5e-6dee-4793-a3b9-02eb12afe784-kube-api-access-zpft7\") on node \"ip-10-0-139-137.ec2.internal\" DevicePath \"\""
Sep 11 05:14:04.177885 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.177852    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" containerID="5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5" exitCode=0
Sep 11 05:14:04.178082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.177894    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"
Sep 11 05:14:04.178082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.177944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" event={"ID":"ba00eb5e-6dee-4793-a3b9-02eb12afe784","Type":"ContainerDied","Data":"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"}
Sep 11 05:14:04.178082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.177986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6" event={"ID":"ba00eb5e-6dee-4793-a3b9-02eb12afe784","Type":"ContainerDied","Data":"3e9c8443e5a3aaa3aef5ebc5b2dc8349e87bdcd17e3675d657d8b17b41815250"}
Sep 11 05:14:04.178082 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.178007    2581 scope.go:117] "RemoveContainer" containerID="5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"
Sep 11 05:14:04.187710 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.187693    2581 scope.go:117] "RemoveContainer" containerID="5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"
Sep 11 05:14:04.187997 ip-10-0-139-137 kubenswrapper[2581]: E0911 05:14:04.187981    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5\": container with ID starting with 5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5 not found: ID does not exist" containerID="5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"
Sep 11 05:14:04.188035 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.188005    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5"} err="failed to get container status \"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5\": rpc error: code = NotFound desc = could not find container \"5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5\": container with ID starting with 5b204793f8747557ac17831c2d578d6558a124f170c74a75358eb63f5668fab5 not found: ID does not exist"
Sep 11 05:14:04.191453 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.191433    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13a06733-5871-4d87-8774-3d5d432af24d" path="/var/lib/kubelet/pods/13a06733-5871-4d87-8774-3d5d432af24d/volumes"
Sep 11 05:14:04.198674 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.198655    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:14:04.203462 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:04.203416    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-60eadfb9-kserve-67b5fcf575-b4vf6"]
Sep 11 05:14:06.192216 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:06.192182    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" path="/var/lib/kubelet/pods/ba00eb5e-6dee-4793-a3b9-02eb12afe784/volumes"
Sep 11 05:14:06.701315 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:06.701287    2581 scope.go:117] "RemoveContainer" containerID="2de4a93b6994b31dea171cdcbeacbdc2051c364fc7c618fa21d7710e0e7739ac"
Sep 11 05:14:06.709971 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:06.709945    2581 scope.go:117] "RemoveContainer" containerID="e086cd1cd9dd46f5b8f279b95cf8c47181d027e09db9c14a796d99d4106f1c82"
Sep 11 05:14:41.615621 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:41.615503    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qjwcz_6434134a-0b1f-4c7f-814c-0ecdd3aeec5d/server/0.log"
Sep 11 05:14:41.746612 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:41.746562    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-76qwm_3bf62e86-0a85-409c-86ea-badf16584a4d/manager/0.log"
Sep 11 05:14:42.140109 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:42.140085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm_e055a6bb-df5d-425c-96c8-fa65d4618e38/copy-manifests/0.log"
Sep 11 05:14:42.157694 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:42.157670    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm_e055a6bb-df5d-425c-96c8-fa65d4618e38/manager/0.log"
Sep 11 05:14:42.718718 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:42.718690    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818380-q2874_39b5fa7c-2438-4531-b8bc-fff54661eafb/cleanup/0.log"
Sep 11 05:14:44.658981 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:44.658947    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg_2769ea72-85d6-44d3-ad7d-dae206e9cf47/istio-proxy/0.log"
Sep 11 05:14:44.906156 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:44.906125    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fcb5f7d79-d492c_942af10e-fe96-4859-be53-7f7b3e9f5407/kube-auth-proxy/0.log"
Sep 11 05:14:45.970637 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:45.970610    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jsjfp_9ccc911f-eaf0-4917-80d4-ae7a2fbf27a3/main/0.log"
Sep 11 05:14:46.315294 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:46.315212    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsbr4f_5f618774-b247-4fa7-888a-1f72eed8f3f9/main/0.log"
Sep 11 05:14:46.425273 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:46.425247    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hp5nn_722d726a-fe13-4a9c-bdab-77d73891ec2d/main/0.log"
Sep 11 05:14:46.550127 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:46.550103    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-fzjkz_423573b6-4e7d-4986-87ef-d0224ee401be/main/0.log"
Sep 11 05:14:52.409005 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.408969    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hb6n7/must-gather-8s65v"]
Sep 11 05:14:52.409382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409305    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a06733-5871-4d87-8774-3d5d432af24d" containerName="main"
Sep 11 05:14:52.409382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409316    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a06733-5871-4d87-8774-3d5d432af24d" containerName="main"
Sep 11 05:14:52.409382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409333    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" containerName="main"
Sep 11 05:14:52.409382 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409338    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" containerName="main"
Sep 11 05:14:52.409524 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a06733-5871-4d87-8774-3d5d432af24d" containerName="main"
Sep 11 05:14:52.409524 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.409405    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba00eb5e-6dee-4793-a3b9-02eb12afe784" containerName="main"
Sep 11 05:14:52.412335 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.412320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.415818 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.415800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hb6n7\"/\"kube-root-ca.crt\""
Sep 11 05:14:52.416118 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.416105    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hb6n7\"/\"openshift-service-ca.crt\""
Sep 11 05:14:52.416204 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.416109    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hb6n7\"/\"default-dockercfg-vqpj6\""
Sep 11 05:14:52.425707 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.425686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/must-gather-8s65v"]
Sep 11 05:14:52.539677 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.539654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/828a7497-f06f-4457-ab10-51d07e06a358-must-gather-output\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.539813 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.539686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktrfc\" (UniqueName: \"kubernetes.io/projected/828a7497-f06f-4457-ab10-51d07e06a358-kube-api-access-ktrfc\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.640430 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.640405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/828a7497-f06f-4457-ab10-51d07e06a358-must-gather-output\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.640537 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.640436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktrfc\" (UniqueName: \"kubernetes.io/projected/828a7497-f06f-4457-ab10-51d07e06a358-kube-api-access-ktrfc\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.640826 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.640804    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/828a7497-f06f-4457-ab10-51d07e06a358-must-gather-output\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.654232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.654205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktrfc\" (UniqueName: \"kubernetes.io/projected/828a7497-f06f-4457-ab10-51d07e06a358-kube-api-access-ktrfc\") pod \"must-gather-8s65v\" (UID: \"828a7497-f06f-4457-ab10-51d07e06a358\") " pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:52.721273 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:52.721253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/must-gather-8s65v"
Sep 11 05:14:53.008664 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:53.008637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/must-gather-8s65v"]
Sep 11 05:14:53.010085 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:14:53.010054    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod828a7497_f06f_4457_ab10_51d07e06a358.slice/crio-49f2157fcdacc59943d1bba058fb806ed2b8492710a5b018ac8803ce9f1c9c1a WatchSource:0}: Error finding container 49f2157fcdacc59943d1bba058fb806ed2b8492710a5b018ac8803ce9f1c9c1a: Status 404 returned error can't find the container with id 49f2157fcdacc59943d1bba058fb806ed2b8492710a5b018ac8803ce9f1c9c1a
Sep 11 05:14:53.012017 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:53.011996    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 05:14:53.424825 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:53.424737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/must-gather-8s65v" event={"ID":"828a7497-f06f-4457-ab10-51d07e06a358","Type":"ContainerStarted","Data":"49f2157fcdacc59943d1bba058fb806ed2b8492710a5b018ac8803ce9f1c9c1a"}
Sep 11 05:14:54.436027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:54.435088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/must-gather-8s65v" event={"ID":"828a7497-f06f-4457-ab10-51d07e06a358","Type":"ContainerStarted","Data":"0c6cf16515d469374df296f9d0bbb0b80d2b3bb0eec07c10e2318726f46eea5a"}
Sep 11 05:14:54.436027 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:54.435143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/must-gather-8s65v" event={"ID":"828a7497-f06f-4457-ab10-51d07e06a358","Type":"ContainerStarted","Data":"4ee5ba969d176527d44a2b4668131ff583e7a32aa9b8f0c0241dfbcdc7bf3697"}
Sep 11 05:14:55.632552 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:55.632526    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-72h62_821978bb-a169-482b-bfde-a70dcbc646d4/global-pull-secret-syncer/0.log"
Sep 11 05:14:55.838145 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:55.838105    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n5vzf_e34a4577-5ec7-4373-8b1d-c6b212d224f9/konnectivity-agent/0.log"
Sep 11 05:14:55.881065 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:14:55.881036    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-137.ec2.internal_4e4442bfe415e79ad545fffb4094f327/haproxy/0.log"
Sep 11 05:15:00.161871 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.161791    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hb6n7/must-gather-8s65v" podStartSLOduration=7.137296771 podStartE2EDuration="8.161766825s" totalImagesPullingTime="1.024470054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:14:54.462629499 +0000 UTC m=+2088.754103936" watchObservedRunningTime="2026-09-11 05:15:00.161766825 +0000 UTC m=+2094.453241263"
Sep 11 05:15:00.162436 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.162081    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"]
Sep 11 05:15:00.168596 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.168552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"
Sep 11 05:15:00.172903 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.172881    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 05:15:00.178596 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.178556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"]
Sep 11 05:15:00.183898 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.183871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"
Sep 11 05:15:00.534132 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:00.533922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"]
Sep 11 05:15:00.537002 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:15:00.536974    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579679c4_3f5b_479d_bcd3_fe58f84d759a.slice/crio-9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d WatchSource:0}: Error finding container 9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d: Status 404 returned error can't find the container with id 9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d
Sep 11 05:15:01.492062 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:01.492020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5" event={"ID":"579679c4-3f5b-479d-bcd3-fe58f84d759a","Type":"ContainerStarted","Data":"fea452ee910b31936f36bafe014bac16b3d395179602790ffbac784467207d71"}
Sep 11 05:15:01.492725 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:01.492692    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5" event={"ID":"579679c4-3f5b-479d-bcd3-fe58f84d759a","Type":"ContainerStarted","Data":"9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d"}
Sep 11 05:15:01.509712 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:01.509645    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5" podStartSLOduration=1.509624839 podStartE2EDuration="1.509624839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:15:01.508825718 +0000 UTC m=+2095.800300155" watchObservedRunningTime="2026-09-11 05:15:01.509624839 +0000 UTC m=+2095.801099279"
Sep 11 05:15:02.324501 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.324470    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-hvtsp_c605a2a6-fbf1-47cf-a5d9-624aedca0a01/cluster-monitoring-operator/0.log"
Sep 11 05:15:02.502163 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.502120    2581 generic.go:356] "Generic (PLEG): container finished" podID="579679c4-3f5b-479d-bcd3-fe58f84d759a" containerID="fea452ee910b31936f36bafe014bac16b3d395179602790ffbac784467207d71" exitCode=0
Sep 11 05:15:02.502654 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.502221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5" event={"ID":"579679c4-3f5b-479d-bcd3-fe58f84d759a","Type":"ContainerDied","Data":"fea452ee910b31936f36bafe014bac16b3d395179602790ffbac784467207d71"}
Sep 11 05:15:02.634628 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.634501    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-crht5_e90d2a02-1fe3-4598-a64e-2cef26c5154f/node-exporter/0.log"
Sep 11 05:15:02.678434 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.678411    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-crht5_e90d2a02-1fe3-4598-a64e-2cef26c5154f/kube-rbac-proxy/0.log"
Sep 11 05:15:02.747641 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:02.747611    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-crht5_e90d2a02-1fe3-4598-a64e-2cef26c5154f/init-textfile/0.log"
Sep 11 05:15:03.731568 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:03.731539    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"
Sep 11 05:15:04.518742 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:04.518668    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5"
Sep 11 05:15:04.518921 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:04.518673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818395-kfrs5" event={"ID":"579679c4-3f5b-479d-bcd3-fe58f84d759a","Type":"ContainerDied","Data":"9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d"}
Sep 11 05:15:04.518921 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:04.518814    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9710ad35d43c4121403118cfe0ec1a24117de5c6c4d971f678ff6e9ee781319d"
Sep 11 05:15:05.743138 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.743102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:15:05.747632 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.747606    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/3.log"
Sep 11 05:15:05.928943 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.928905    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"]
Sep 11 05:15:05.929285 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.929267    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="579679c4-3f5b-479d-bcd3-fe58f84d759a" containerName="cleanup"
Sep 11 05:15:05.929361 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.929287    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="579679c4-3f5b-479d-bcd3-fe58f84d759a" containerName="cleanup"
Sep 11 05:15:05.929422 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.929395    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="579679c4-3f5b-479d-bcd3-fe58f84d759a" containerName="cleanup"
Sep 11 05:15:05.934476 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.934450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:05.953215 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:05.953184    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"]
Sep 11 05:15:06.085976 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.085894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mbpj\" (UniqueName: \"kubernetes.io/projected/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-kube-api-access-7mbpj\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.086152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.085995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-lib-modules\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.086152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.086023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-sys\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.086152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.086055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-podres\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.086152 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.086101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-proc\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.186936 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.186891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-proc\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187129 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.186951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mbpj\" (UniqueName: \"kubernetes.io/projected/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-kube-api-access-7mbpj\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187129 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-proc\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187129 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-lib-modules\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187129 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-sys\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187129 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-podres\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187415 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-lib-modules\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187415 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-sys\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.187415 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.187324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-podres\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.203089 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.203059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mbpj\" (UniqueName: \"kubernetes.io/projected/d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc-kube-api-access-7mbpj\") pod \"perf-node-gather-daemonset-v2799\" (UID: \"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.248299 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.248266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:06.380124 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.380050    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:15:06.395016 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.394771    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-tzsh8_c127bc70-006d-4bf8-adab-0d937a3b5102/console-operator/2.log"
Sep 11 05:15:06.572904 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.572881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"]
Sep 11 05:15:06.575798 ip-10-0-139-137 kubenswrapper[2581]: W0911 05:15:06.575770    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd318a3e8_9463_4c80_9bb1_ffab6ca9f3cc.slice/crio-2b06624016ed04c1e1850ab1d4ec63cb1aaaffa43e740a3bda2f54ecad201a2e WatchSource:0}: Error finding container 2b06624016ed04c1e1850ab1d4ec63cb1aaaffa43e740a3bda2f54ecad201a2e: Status 404 returned error can't find the container with id 2b06624016ed04c1e1850ab1d4ec63cb1aaaffa43e740a3bda2f54ecad201a2e
Sep 11 05:15:06.740396 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:06.740362    2581 scope.go:117] "RemoveContainer" containerID="b39f7bc0f60f2f2234320380a01f75b69f0f721801c9a8bae54502f1af94702b"
Sep 11 05:15:07.323256 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:07.323228    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-kb9gp_276dc052-1ce1-4884-ae66-788ebe3d5fd8/volume-data-source-validator/0.log"
Sep 11 05:15:07.542551 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:07.542515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799" event={"ID":"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc","Type":"ContainerStarted","Data":"7b5d2b75c2861abe80297ff5638c67cad786fdbd25402c2cfc891f6846554f43"}
Sep 11 05:15:07.542736 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:07.542593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799" event={"ID":"d318a3e8-9463-4c80-9bb1-ffab6ca9f3cc","Type":"ContainerStarted","Data":"2b06624016ed04c1e1850ab1d4ec63cb1aaaffa43e740a3bda2f54ecad201a2e"}
Sep 11 05:15:07.542736 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:07.542630    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:07.569467 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:07.569408    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799" podStartSLOduration=2.5693919259999998 podStartE2EDuration="2.569391926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 05:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:15:07.564782394 +0000 UTC m=+2101.856256833" watchObservedRunningTime="2026-09-11 05:15:07.569391926 +0000 UTC m=+2101.860866362"
Sep 11 05:15:08.503210 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:08.503177    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-42vp6_a78ba00e-4e12-4712-9f78-408d47756b56/dns/0.log"
Sep 11 05:15:08.538442 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:08.538419    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-42vp6_a78ba00e-4e12-4712-9f78-408d47756b56/kube-rbac-proxy/0.log"
Sep 11 05:15:08.832604 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:08.832523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-md27b_dd6ab5b1-010f-470d-aed1-b35adf58e938/dns-node-resolver/0.log"
Sep 11 05:15:09.605260 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:09.605223    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2wtlv_ddecf3f4-042d-4f9f-b94e-c9deff6d68b0/node-ca/0.log"
Sep 11 05:15:10.766953 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:10.766905    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg_2769ea72-85d6-44d3-ad7d-dae206e9cf47/istio-proxy/0.log"
Sep 11 05:15:11.004995 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:11.004965    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fcb5f7d79-d492c_942af10e-fe96-4859-be53-7f7b3e9f5407/kube-auth-proxy/0.log"
Sep 11 05:15:12.190232 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.190200    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xtll9_6fa39515-1242-4d0e-a11d-4218cfa55fb9/serve-healthcheck-canary/0.log"
Sep 11 05:15:12.664495 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.664463    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jm555_364e817b-78a5-4e76-8fb8-6ac3644bbe8b/insights-operator/0.log"
Sep 11 05:15:12.666947 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.666924    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jm555_364e817b-78a5-4e76-8fb8-6ac3644bbe8b/insights-operator/1.log"
Sep 11 05:15:12.686376 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.686355    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qwrd_be40e39b-4930-4de4-a1f9-43da0959c7b7/kube-rbac-proxy/0.log"
Sep 11 05:15:12.705781 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.705759    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qwrd_be40e39b-4930-4de4-a1f9-43da0959c7b7/exporter/0.log"
Sep 11 05:15:12.727985 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:12.727950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qwrd_be40e39b-4930-4de4-a1f9-43da0959c7b7/extractor/0.log"
Sep 11 05:15:13.560314 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:13.560286    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-v2799"
Sep 11 05:15:15.533793 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.533704    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg_2769ea72-85d6-44d3-ad7d-dae206e9cf47/istio-proxy/0.log"
Sep 11 05:15:15.625470 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.625440    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qjwcz_6434134a-0b1f-4c7f-814c-0ecdd3aeec5d/server/0.log"
Sep 11 05:15:15.729002 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.728941    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-76qwm_3bf62e86-0a85-409c-86ea-badf16584a4d/manager/0.log"
Sep 11 05:15:15.754472 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.754447    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fcb5f7d79-d492c_942af10e-fe96-4859-be53-7f7b3e9f5407/kube-auth-proxy/0.log"
Sep 11 05:15:15.950915 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.950880    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm_e055a6bb-df5d-425c-96c8-fa65d4618e38/manager/0.log"
Sep 11 05:15:15.984850 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:15.984824    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-rzwkm_e055a6bb-df5d-425c-96c8-fa65d4618e38/copy-manifests/0.log"
Sep 11 05:15:17.269233 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:17.269202    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f84bb6f84-bwdnj_0344db76-659e-4118-9702-564aa962fd50/manager/0.log"
Sep 11 05:15:17.331614 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:17.331561    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4hnrk_4c507738-ded7-4541-9fe0-1cfc623064af/openshift-lws-operator/0.log"
Sep 11 05:15:17.372001 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:17.371967    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nn9tg_2769ea72-85d6-44d3-ad7d-dae206e9cf47/istio-proxy/0.log"
Sep 11 05:15:17.613655 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:17.613562    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fcb5f7d79-d492c_942af10e-fe96-4859-be53-7f7b3e9f5407/kube-auth-proxy/0.log"
Sep 11 05:15:22.716064 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:22.716028    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-7mqwk_a7a4371a-2787-4d9f-870e-e7c137a75ed7/kube-storage-version-migrator-operator/1.log"
Sep 11 05:15:22.717195 ip-10-0-139-137 kubenswrapper[2581]: I0911 05:15:22.717152    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-7mqwk_a7a4371a-2787-4d9f-870e-e7c137a75ed7/kube-storage-version-migrator-operator/0.log"