Sep 16 15:50:01.266149 ip-10-0-137-111 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 15:50:01.693148 ip-10-0-137-111 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 15:50:01.693148 ip-10-0-137-111 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 15:50:01.693148 ip-10-0-137-111 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 15:50:01.693148 ip-10-0-137-111 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 15:50:01.693148 ip-10-0-137-111 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 15:50:01.694244 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.694150    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 15:50:01.699347 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699329    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699350    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699354    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699357    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699360    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699364    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699367    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699371    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699376    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699379    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699382    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699385    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699393    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.699391 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699396    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699400    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699402    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699405    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699408    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699410    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699413    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699416    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699418    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699421    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699424    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699427    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699429    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699432    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699436    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699439    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699442    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699445    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699447    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.699707 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699451    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699454    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699457    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699460    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699462    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699465    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699468    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699470    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699473    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699475    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699478    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699480    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699483    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699485    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699488    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699492    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699495    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699498    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699501    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699503    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.700173 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699506    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699508    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699511    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699514    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699516    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699518    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699521    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699524    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699526    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699529    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699532    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699535    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699537    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699540    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699542    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699545    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699547    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699550    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699553    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699555    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.700669 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699558    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699561    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699564    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699567    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699569    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699572    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699574    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699578    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699581    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699584    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699587    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699590    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699592    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699595    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699597    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699601    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699604    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699607    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699610    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.701251 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699612    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699615    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699618    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699620    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699623    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699626    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699628    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.699631    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700386    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700393    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700396    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700399    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700401    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700404    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700406    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700410    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700412    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700415    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700418    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.701739 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700420    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700424    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700427    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700430    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700432    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700435    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700437    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700440    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700443    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700448    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700452    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700454    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700457    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700460    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700463    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700466    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700469    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700472    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700477    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700481    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.702215 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700483    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700486    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700489    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700491    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700494    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700497    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700499    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700502    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700505    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700507    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700510    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700512    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700515    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700520    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700523    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700528    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700531    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700534    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700536    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700539    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.702726 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700542    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700544    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700547    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700549    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700552    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700554    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700557    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700559    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700563    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700567    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700570    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700574    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700577    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700580    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700582    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700585    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700588    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700591    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700593    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.703253 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700596    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700598    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700601    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700604    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700608    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700611    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700616    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700619    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700621    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700624    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700627    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700630    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700633    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700636    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700638    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700641    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700643    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700646    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700649    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700651    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.703715 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700654    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700657    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700659    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700662    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700665    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700668    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700670    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700673    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.700676    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701813    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701820    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701824    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701827    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701830    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701833    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701836    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701839    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701842    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701846    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.704237 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701849    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701852    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701854    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701857    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701859    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701862    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701865    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701867    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701870    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701872    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701875    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701878    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701881    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701883    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701886    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701889    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701891    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701894    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701897    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.704730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701900    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701903    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701905    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701908    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701911    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701915    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701918    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701921    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701923    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701926    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701928    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701930    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701934    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701936    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701940    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701944    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701947    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701950    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701952    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.705206 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701955    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701958    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701960    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701963    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701965    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701967    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701970    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701973    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701975    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701978    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701980    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701983    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701986    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701990    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701993    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701996    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.701999    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702002    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702005    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702008    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.705660 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702012    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702015    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702017    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702020    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702022    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702025    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702028    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702030    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702033    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702035    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702038    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702041    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702043    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702052    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702055    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702057    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702060    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702062    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702064    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702067    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.706195 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702069    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702072    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702074    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702077    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702079    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702082    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702090    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702092    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702095    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702098    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702100    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702173    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702207    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702213    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702218    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702222    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702226    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702231    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702235    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702238    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702241    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702245    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 15:50:01.706692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702248    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702252    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702255    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702257    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702261    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702263    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702266    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702274    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702277    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702280    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702283    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702286    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702290    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702293    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702296    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702299    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702302    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702311    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702314    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702317    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702320    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702325    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702329    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702334    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702339    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 15:50:01.707292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702343    2581 flags.go:64] FLAG: --enable-server="true"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702353    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702361    2581 flags.go:64] FLAG: --event-burst="100"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702365    2581 flags.go:64] FLAG: --event-qps="50"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702368    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702371    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702374    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702378    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702381    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702384    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702387    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702390    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702393    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702396    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702399    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702402    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702406    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702408    2581 flags.go:64] FLAG: --feature-gates=""
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702413    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702417    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702422    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702427    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702433    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702438    2581 flags.go:64] FLAG: --help="false"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702441    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-137-111.ec2.internal"
Sep 16 15:50:01.707913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702446    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702451    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702454    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702458    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702461    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702464    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702467    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702470    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702474    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702477    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702480    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702483    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702486    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702504    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702526    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702554    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702560    2581 flags.go:64] FLAG: --lock-file=""
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702565    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702569    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702573    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702580    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702583    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702587    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 15:50:01.708547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702590    2581 flags.go:64] FLAG: --logging-format="text"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702593    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702597    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702601    2581 flags.go:64] FLAG: --manifest-url=""
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702604    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702609    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702613    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702618    2581 flags.go:64] FLAG: --max-pods="110"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702621    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702624    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702627    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702631    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702634    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702637    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702641    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702649    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702653    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702656    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702659    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702663    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702669    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702672    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702676    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702679    2581 flags.go:64] FLAG: --port="10250"
Sep 16 15:50:01.709138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702682    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702686    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0793a77c9a809b05b"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702690    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702694    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702697    2581 flags.go:64] FLAG: --register-node="true"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702700    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702704    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702707    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702710    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702713    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702717    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702720    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702723    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702726    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702729    2581 flags.go:64] FLAG: --runonce="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702733    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702736    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702740    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702743    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702746    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702749    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702753    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702756    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702759    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702762    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 15:50:01.709724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702765    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702768    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702771    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702775    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702778    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702796    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702800    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702803    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702809    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702812    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702815    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702818    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702821    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702825    2581 flags.go:64] FLAG: --v="2"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702829    2581 flags.go:64] FLAG: --version="false"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702833    2581 flags.go:64] FLAG: --vmodule=""
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702837    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.702841    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702971    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702975    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702979    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702982    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702986    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702989    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.710377 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702992    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702995    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.702998    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703004    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703007    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703010    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703013    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703016    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703019    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703021    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703024    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703027    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703029    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703032    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703035    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703037    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703040    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703043    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703046    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703048    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.711014 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703051    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703054    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703057    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703060    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703062    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703065    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703067    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703070    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703073    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703076    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703079    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703081    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703084    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703087    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703089    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703094    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703096    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703100    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703103    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703105    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.711510 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703108    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703111    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703115    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703119    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703122    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703125    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703128    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703131    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703135    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703139    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703143    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703147    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703150    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703153    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703156    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703159    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703162    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703164    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703167    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.712038 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703169    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703172    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703175    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703178    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703181    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703183    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703186    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703189    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703193    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703196    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703200    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703203    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703206    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703208    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703211    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703213    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703216    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703219    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703221    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703224    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.712810 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703227    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703229    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703232    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703235    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703237    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703240    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703243    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703246    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703248    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703251    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703253    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703256    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703259    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.703261    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.713578 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.704016    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 15:50:01.714110 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714088    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 15:50:01.714110 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714111    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 15:50:01.714170 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714131    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 15:50:01.714170 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714137    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714195    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714200    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714203    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714206    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714210    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714213    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714216    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714219    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714222    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714225    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714227    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714230    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714233    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.714229 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714236    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714239    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714242    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714244    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714247    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714250    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714253    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714255    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714258    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714260    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714263    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714266    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714268    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714271    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714274    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714278    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714282    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714285    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714288    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.714560 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714290    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714293    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714295    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714298    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714300    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714303    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714305    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714308    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714311    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714313    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714316    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714318    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714321    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714324    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714327    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714330    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714333    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714336    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714338    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714341    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.715079 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714343    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714346    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714349    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714352    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714354    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714357    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714359    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714362    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714365    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714368    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714371    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714373    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714376    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714378    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714382    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714384    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714387    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714390    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714392    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714395    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.715579 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714398    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714400    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714403    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714405    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714408    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714411    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714414    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714417    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714420    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714425    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714429    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714432    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714435    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714438    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714441    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714444    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714447    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714450    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714453    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.716103 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714455    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714459    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714463    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714466    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714469    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714472    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714475    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714478    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714483    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714596    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714601    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714604    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714607    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714610    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 15:50:01.716558 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714613    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714615    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714618    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714621    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714623    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714626    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714629    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714633    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714636    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714638    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714641    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714644    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714646    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714654    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714657    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714659    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714662    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714665    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714669    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 15:50:01.716922 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714672    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714675    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714678    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714681    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714684    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714687    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714690    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714692    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714695    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714698    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714702    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714704    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714707    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714710    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714713    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714715    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714718    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714720    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714723    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714726    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 15:50:01.717383 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714729    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714732    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714734    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714737    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714740    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714742    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714745    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714748    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714750    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714753    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714755    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714758    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714761    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714763    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714766    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714768    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714771    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714774    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714776    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714793    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 15:50:01.717879 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714798    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714801    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714804    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714806    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714809    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714812    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714814    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714817    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714819    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714822    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714825    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714828    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714831    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714833    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714838    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714841    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714843    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714846    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714848    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714851    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 15:50:01.718387 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714853    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714856    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714858    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714862    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714866    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714869    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714871    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714874    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714877    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714880    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714882    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714885    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714887    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714890    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:01.714892    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 15:50:01.718975 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.714897    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 15:50:01.719337 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.715014    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 15:50:01.720304 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.720287    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 15:50:01.721425 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.721412    2581 server.go:1019] "Starting client certificate rotation"
Sep 16 15:50:01.721544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.721525    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 15:50:01.721606 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.721572    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 15:50:01.724395 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.724379    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 15:50:01.725581 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.725562    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 15:50:01.736818 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.736798    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 16 15:50:01.742221 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.742202    2581 log.go:25] "Validated CRI v1 image API"
Sep 16 15:50:01.743010 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.742989    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 15:50:01.748086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.748062    2581 fs.go:135] Filesystem UUIDs: map[5219f8f2-0c1c-48d6-9135-e6510732e14b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f843ad84-5d3d-4b9a-bddc-933705aba3fc:/dev/nvme0n1p3]
Sep 16 15:50:01.748165 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.748085    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 15:50:01.755238 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.754837    2581 manager.go:216] Machine: {Timestamp:2026-09-16 15:50:01.753019161 +0000 UTC m=+0.381852335 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3083116 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21d37cef99216485242db8197cc6fc SystemUUID:ec21d37c-ef99-2164-8524-2db8197cc6fc BootID:eda123e1-431d-4c42-b0a0-f3d552b1b803 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fb:04:e5:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fb:04:e5:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:04:3c:83:38:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 15:50:01.755238 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.755221    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 15:50:01.755393 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.755353    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 15:50:01.755919 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.755903    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 15:50:01.757322 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.757292    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 15:50:01.757459 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.757325    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-111.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 15:50:01.758154 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.758144    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 15:50:01.758190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.758157    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 15:50:01.758190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.758166    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 15:50:01.758986 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.758975    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 15:50:01.760263 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.760253    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 15:50:01.764985 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.764969    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 15:50:01.765112 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765102    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 15:50:01.765145 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765134    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 15:50:01.765145 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765144    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 15:50:01.765196 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765158    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 15:50:01.765196 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765168    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 16 15:50:01.765196 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765178    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 15:50:01.765613 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765594    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 15:50:01.765710 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.765618    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 15:50:01.768074 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.768058    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 15:50:01.771074 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.771059    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 15:50:01.771148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.771090    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 15:50:01.773168 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773153    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 15:50:01.773168 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773171    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773177    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773183    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773192    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773198    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773211    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773217    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773224    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773230    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773245    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 15:50:01.773255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773252    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 15:50:01.773941 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773928    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 15:50:01.773941 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.773938    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 15:50:01.775178 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.775148    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 15:50:01.775317 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.775281    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-111.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 15:50:01.776942 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.776928    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-111.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 15:50:01.778029 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.778016    2581 server.go:1268] "Started kubelet"
Sep 16 15:50:01.778105 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.778078    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 15:50:01.778360 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.778190    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 15:50:01.778360 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.778277    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 15:50:01.779046 ip-10-0-137-111 systemd[1]: Started Kubernetes Kubelet.
Sep 16 15:50:01.779736 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.779687    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 15:50:01.780469 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.780446    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 15:50:01.781280 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.781247    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7m45q"
Sep 16 15:50:01.782690 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.782671    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 15:50:01.788264 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.788246    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7m45q"
Sep 16 15:50:01.788350 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.787155    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-111.ec2.internal.18d5d7e422528550  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-111.ec2.internal,UID:ip-10-0-137-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-111.ec2.internal,},FirstTimestamp:2026-09-16 15:50:01.777988944 +0000 UTC m=+0.406822113,LastTimestamp:2026-09-16 15:50:01.777988944 +0000 UTC m=+0.406822113,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-111.ec2.internal,}"
Sep 16 15:50:01.795484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.795468    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 15:50:01.796109 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796088    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 15:50:01.796222 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.796203    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:01.796826 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796769    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 15:50:01.796826 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796770    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 15:50:01.796826 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796819    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 15:50:01.797010 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796938    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 15:50:01.797010 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.796944    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 15:50:01.797962 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.797944    2581 factory.go:55] Registering systemd factory
Sep 16 15:50:01.798062 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.797973    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 16 15:50:01.798062 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.797970    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-111.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 15:50:01.798250 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798237    2581 factory.go:153] Registering CRI-O factory
Sep 16 15:50:01.798316 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798254    2581 factory.go:223] Registration of the crio container factory successfully
Sep 16 15:50:01.798364 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798321    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 15:50:01.798364 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798340    2581 factory.go:103] Registering Raw factory
Sep 16 15:50:01.798453 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798374    2581 manager.go:1196] Started watching for new ooms in manager
Sep 16 15:50:01.798733 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.798720    2581 manager.go:349] Starting recovery of all containers
Sep 16 15:50:01.810644 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.810625    2581 manager.go:354] Recovery completed
Sep 16 15:50:01.811130 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.811109    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:01.814600 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.814588    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:01.816147 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816129    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:01.816227 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816160    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:01.816227 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816170    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:01.816749 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816733    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 15:50:01.816749 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816748    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 15:50:01.816900 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816766    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 15:50:01.816900 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.816821    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:01.817507 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.817492    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:01.817561 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.817522    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:01.817561 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.817532    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:01.818909 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.818896    2581 policy_none.go:49] "None policy: Start"
Sep 16 15:50:01.818977 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.818915    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 15:50:01.818977 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.818928    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 15:50:01.820973 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.820960    2581 policy_none.go:47] "Start"
Sep 16 15:50:01.859653 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.859631    2581 manager.go:337] "Starting Device Plugin manager"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.859682    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.859698    2581 server.go:87] "Starting device plugin registration server"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.860177    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.860197    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.860290    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.860366    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.860374    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.860920    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 15:50:01.875345 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.860987    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:01.900891 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.900855    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 15:50:01.902077 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.902053    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 15:50:01.902167 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.902115    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 15:50:01.902218 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.902170    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 15:50:01.902336 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.902293    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 15:50:01.904845 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.904826    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:01.961296 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.961175    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:01.962297 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.962281    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:01.962382 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.962316    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:01.962382 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.962334    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:01.962382 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.962356    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-111.ec2.internal"
Sep 16 15:50:01.968526 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:01.968511    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-111.ec2.internal"
Sep 16 15:50:01.968597 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.968532    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-111.ec2.internal\": node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:01.983568 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:01.983529    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:02.003167 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.003138    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"]
Sep 16 15:50:02.003271 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.003207    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:02.005194 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.005170    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:02.005298 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.005203    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:02.005298 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.005223    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:02.007753 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.007738    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:02.007890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.007861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.007939 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.007894    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:02.008542 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008523    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:02.008623 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008555    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:02.008623 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008584    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:02.008623 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008601    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:02.008715 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008559    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:02.008715 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.008652    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:02.010953 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.010939    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.011028 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.010962    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 15:50:02.011646 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.011625    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 15:50:02.011724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.011650    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 15:50:02.011724 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.011663    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeHasSufficientPID"
Sep 16 15:50:02.030427 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:02.030408    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-111.ec2.internal\" not found" node="ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.034871 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:02.034855    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-111.ec2.internal\" not found" node="ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.083683 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:02.083653    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:02.098641 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.098618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.098743 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.098647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b7775f07a812f020e26327f5b0a385a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-111.ec2.internal\" (UID: \"6b7775f07a812f020e26327f5b0a385a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.098743 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.098664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.184038 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:02.183995    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:02.199412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b7775f07a812f020e26327f5b0a385a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-111.ec2.internal\" (UID: \"6b7775f07a812f020e26327f5b0a385a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.199412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.199573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.199573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.199573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6b7775f07a812f020e26327f5b0a385a-config\") pod \"kube-apiserver-proxy-ip-10-0-137-111.ec2.internal\" (UID: \"6b7775f07a812f020e26327f5b0a385a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.199573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:02.199501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3d42f42d8f0ded147a5353becd15c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal\" (UID: \"07b3d42f42d8f0ded147a5353becd15c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:02.284914 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:02.284827    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:03.734670 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:03.734641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:03.738440 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:03.738418    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:03.787203 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:03.787144    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:03.887666 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:03.887630    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:03.988213 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:03.988102    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.088630 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.088601    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.123071 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.123046    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 15:50:04.123484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.123205    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 15:50:04.123484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.123226    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 15:50:04.189237 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.189209    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.198757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.198717    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 15:45:03 +0000 UTC" deadline="2028-07-01 05:46:28.616470348 +0000 UTC"
Sep 16 15:50:04.198757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.198750    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15685h56m24.417724106s"
Sep 16 15:50:04.289718 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.289634    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.344471 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:04.344418    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b3d42f42d8f0ded147a5353becd15c.slice/crio-c928692fbfabdb2c7e18da68df19aac2ba81a395159a028d2c44efb6aced8cf3 WatchSource:0}: Error finding container c928692fbfabdb2c7e18da68df19aac2ba81a395159a028d2c44efb6aced8cf3: Status 404 returned error can't find the container with id c928692fbfabdb2c7e18da68df19aac2ba81a395159a028d2c44efb6aced8cf3
Sep 16 15:50:04.344745 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:04.344716    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7775f07a812f020e26327f5b0a385a.slice/crio-b7f2d0d0f83c1ca2dc9e238b13973bad90c80d20d49255de723d2e6f996ec136 WatchSource:0}: Error finding container b7f2d0d0f83c1ca2dc9e238b13973bad90c80d20d49255de723d2e6f996ec136: Status 404 returned error can't find the container with id b7f2d0d0f83c1ca2dc9e238b13973bad90c80d20d49255de723d2e6f996ec136
Sep 16 15:50:04.348316 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.348299    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 15:50:04.390433 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.390379    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.478352 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.478320    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:04.491463 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.491437    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.592037 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.591949    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.692783 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:04.692745    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-111.ec2.internal\" not found"
Sep 16 15:50:04.750924 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.750894    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:04.798285 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.798261    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:04.808849 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.808829    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 15:50:04.809588 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.809574    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal"
Sep 16 15:50:04.817884 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.817866    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 15:50:04.933207 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:04.933127    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:05.149604 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.149568    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 15:50:05.167706 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.167675    2581 apiserver.go:52] "Watching apiserver"
Sep 16 15:50:05.175852 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.175822    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 15:50:05.176217 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.176195    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-lnfvq","kube-system/global-pull-secret-syncer-hsntg","kube-system/konnectivity-agent-krlk5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal","openshift-multus/network-metrics-daemon-rv47s","openshift-network-operator/iptables-alerter-k76fx","kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal","openshift-cluster-node-tuning-operator/tuned-z7txb","openshift-dns/node-resolver-rhqrq","openshift-image-registry/node-ca-v7nh6","openshift-multus/multus-additional-cni-plugins-xzzp9","openshift-multus/multus-x4nbv","openshift-network-diagnostics/network-check-target-2xlht"]
Sep 16 15:50:05.178732 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.178713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.180966 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.180937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.181550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.181529    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fj2tn\""
Sep 16 15:50:05.181650 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.181551    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.181650 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.181552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.181729 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.181665    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 15:50:05.183379 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.183020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.183471 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.183417    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 15:50:05.183710 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.183624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xjcsb\""
Sep 16 15:50:05.183710 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.183656    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.183898 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.183876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.185295 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.185276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.185789 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.185769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fmjgz\""
Sep 16 15:50:05.185923 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.185903    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 15:50:05.185992 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.185913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 15:50:05.187459 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.187439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:05.187539 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.187519    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:05.187795 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.187778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.187859 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.187838    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 15:50:05.188016 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.187999    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.188136 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.188117    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rs9mr\""
Sep 16 15:50:05.190021 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.190003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.190178 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.190075    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:05.192400 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.192385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.194640 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.194619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.195437 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.195413    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.195756 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.195736    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n6dwg\""
Sep 16 15:50:05.195832 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.195779    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.199216 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.197689    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 15:50:05.199592 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.199508    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 15:50:05.199592 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.199571    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.199940 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.199917    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9dmzp\""
Sep 16 15:50:05.200033 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.199993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.200132 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.200114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 15:50:05.200560 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.200542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.200641 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.200607    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:05.202818 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.202799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.205291 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.205266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.205389 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.205319    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zdsmg\""
Sep 16 15:50:05.205567 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.205274    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.205625 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.205271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.207723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.207708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.207937 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.207916    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 15:50:05.208062 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.208045    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 15:50:05.208249 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.208234    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6tmhl\""
Sep 16 15:50:05.208347 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.208333    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 15:50:05.209167 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.209140    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 15:50:05.209462 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.209442    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 15:50:05.209595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.209442    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 15:50:05.209986 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.209968    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 15:50:05.210048 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.209990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-hostroot\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210048 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-dbus\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.210148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.210148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f32d48d7-ec36-4a76-8782-071398693244-iptables-alerter-script\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.210148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb09db2-2a8a-48a2-8270-ac314de8cc96-host\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.210313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-kubernetes\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-748rb\""
Sep 16 15:50:05.210313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-conf\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-system-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-daemon-config\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a760ae3f-642b-4210-b55b-6809037b13fa-agent-certs\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-sys\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-var-lib-kubelet\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-tmp\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-socket-dir-parent\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210476    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 15:50:05.210550 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f32d48d7-ec36-4a76-8782-071398693244-host-slash\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1cb09db2-2a8a-48a2-8270-ac314de8cc96-serviceca\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a760ae3f-642b-4210-b55b-6809037b13fa-konnectivity-ca\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-cni-binary-copy\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6blnx\" (UniqueName: \"kubernetes.io/projected/1cb09db2-2a8a-48a2-8270-ac314de8cc96-kube-api-access-6blnx\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-socket-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-device-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-modprobe-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-lib-modules\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-netns\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-multus-certs\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdngb\" (UniqueName: \"kubernetes.io/projected/f32d48d7-ec36-4a76-8782-071398693244-kube-api-access-xdngb\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:05.210906 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysconfig\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-systemd\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx8f5\" (UniqueName: \"kubernetes.io/projected/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-kube-api-access-mx8f5\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-multus\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.210994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-kubelet\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-etc-kubernetes\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-kubelet-config\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211064    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-registration-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpjs\" (UniqueName: \"kubernetes.io/projected/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-kube-api-access-8kpjs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-tuned\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-conf-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-sys-fs\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssz9n\" (UniqueName: \"kubernetes.io/projected/f8369695-3f94-44ef-8f93-cf5bf167e97b-kube-api-access-ssz9n\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-run\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.211436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-host\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-bin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82z55\" (UniqueName: \"kubernetes.io/projected/4621b60a-f864-49f5-bec2-0b28d89a9e89-kube-api-access-82z55\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-cnibin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-k8s-cni-cncf-io\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.212036 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.211748    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 15:50:05.229591 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.229555    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6gr7h"
Sep 16 15:50:05.235468 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.235443    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6gr7h"
Sep 16 15:50:05.300018 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.299974    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 15:50:05.308085 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.308028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal" event={"ID":"6b7775f07a812f020e26327f5b0a385a","Type":"ContainerStarted","Data":"b7f2d0d0f83c1ca2dc9e238b13973bad90c80d20d49255de723d2e6f996ec136"}
Sep 16 15:50:05.309139 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.309109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal" event={"ID":"07b3d42f42d8f0ded147a5353becd15c","Type":"ContainerStarted","Data":"c928692fbfabdb2c7e18da68df19aac2ba81a395159a028d2c44efb6aced8cf3"}
Sep 16 15:50:05.312428 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-bin\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-socket-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-modprobe-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-lib-modules\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-netns\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-multus-certs\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdngb\" (UniqueName: \"kubernetes.io/projected/f32d48d7-ec36-4a76-8782-071398693244-kube-api-access-xdngb\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.312535 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6a2a639-00a1-46a0-a895-bb7f6c89c275-hosts-file\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-log-socket\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-system-cni-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-env-overrides\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-script-lib\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-tuned\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-slash\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-sys-fs\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssz9n\" (UniqueName: \"kubernetes.io/projected/f8369695-3f94-44ef-8f93-cf5bf167e97b-kube-api-access-ssz9n\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-host\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-bin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82z55\" (UniqueName: \"kubernetes.io/projected/4621b60a-f864-49f5-bec2-0b28d89a9e89-kube-api-access-82z55\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-dbus\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f32d48d7-ec36-4a76-8782-071398693244-iptables-alerter-script\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.312877 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-var-lib-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-cnibin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-k8s-cni-cncf-io\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-etc-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-config\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovn-node-metrics-cert\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s54j\" (UniqueName: \"kubernetes.io/projected/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-kube-api-access-4s54j\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-kubernetes\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.312999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26xtg\" (UniqueName: \"kubernetes.io/projected/b6a2a639-00a1-46a0-a895-bb7f6c89c275-kube-api-access-26xtg\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a760ae3f-642b-4210-b55b-6809037b13fa-agent-certs\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-var-lib-kubelet\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-tmp\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-k8s-cni-cncf-io\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f32d48d7-ec36-4a76-8782-071398693244-host-slash\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.313515 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-bin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-modprobe-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1cb09db2-2a8a-48a2-8270-ac314de8cc96-serviceca\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6blnx\" (UniqueName: \"kubernetes.io/projected/1cb09db2-2a8a-48a2-8270-ac314de8cc96-kube-api-access-6blnx\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-dbus\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-cnibin\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a760ae3f-642b-4210-b55b-6809037b13fa-konnectivity-ca\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-socket-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-var-lib-kubelet\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-netns\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-run-multus-certs\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-lib-modules\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-sys-fs\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.313627    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f32d48d7-ec36-4a76-8782-071398693244-host-slash\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.314346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-host\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.313718    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:05.813673576 +0000 UTC m=+3.041131586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-kubernetes\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-cni-binary-copy\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313861    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-kubelet\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.313993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-d\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-node-log\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-device-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysconfig\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314279    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-device-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysconfig\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-systemd\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx8f5\" (UniqueName: \"kubernetes.io/projected/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-kube-api-access-mx8f5\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-multus\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-kubelet\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-cni-binary-copy\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-etc-kubernetes\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-etc-kubernetes\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-kubelet-config\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-registration-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpjs\" (UniqueName: \"kubernetes.io/projected/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-kube-api-access-8kpjs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-conf-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-systemd\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f32d48d7-ec36-4a76-8782-071398693244-iptables-alerter-script\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/08ec670f-7702-4bae-a7c9-2e94c3b511b5-kubelet-config\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-kubelet\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314619    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-registration-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-host-var-lib-cni-multus\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.315794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-conf-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6p6h\" (UniqueName: \"kubernetes.io/projected/0bdb496b-80af-482b-bb44-661de8ec606c-kube-api-access-w6p6h\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a760ae3f-642b-4210-b55b-6809037b13fa-konnectivity-ca\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a2a639-00a1-46a0-a895-bb7f6c89c275-tmp-dir\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-run\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-hostroot\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb09db2-2a8a-48a2-8270-ac314de8cc96-host\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-run\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8369695-3f94-44ef-8f93-cf5bf167e97b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-hostroot\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.314998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-netns\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb09db2-2a8a-48a2-8270-ac314de8cc96-host\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-ovn\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.315036    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:05.316617 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-conf\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.315107    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:05.815095147 +0000 UTC m=+3.042553131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-system-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-daemon-config\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-cnibin\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-sysctl-conf\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-systemd-units\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315282    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-system-cni-dir\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-systemd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-sys\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-socket-dir-parent\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-netd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-socket-dir-parent\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-sys\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.315735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4621b60a-f864-49f5-bec2-0b28d89a9e89-multus-daemon-config\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.317480 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.316458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1cb09db2-2a8a-48a2-8270-ac314de8cc96-serviceca\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.318049 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.317720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-tmp\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.318049 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.317791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-etc-tuned\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.318144 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.318116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a760ae3f-642b-4210-b55b-6809037b13fa-agent-certs\") pod \"konnectivity-agent-krlk5\" (UID: \"a760ae3f-642b-4210-b55b-6809037b13fa\") " pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.322795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.322760    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:05.322795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.322782    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:05.322795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.322793    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:05.323040 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.322852    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:05.822837599 +0000 UTC m=+3.050295584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:05.323444 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.323414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82z55\" (UniqueName: \"kubernetes.io/projected/4621b60a-f864-49f5-bec2-0b28d89a9e89-kube-api-access-82z55\") pod \"multus-x4nbv\" (UID: \"4621b60a-f864-49f5-bec2-0b28d89a9e89\") " pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.324841 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.324802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdngb\" (UniqueName: \"kubernetes.io/projected/f32d48d7-ec36-4a76-8782-071398693244-kube-api-access-xdngb\") pod \"iptables-alerter-k76fx\" (UID: \"f32d48d7-ec36-4a76-8782-071398693244\") " pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.324987 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.324970    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssz9n\" (UniqueName: \"kubernetes.io/projected/f8369695-3f94-44ef-8f93-cf5bf167e97b-kube-api-access-ssz9n\") pod \"aws-ebs-csi-driver-node-qlzws\" (UID: \"f8369695-3f94-44ef-8f93-cf5bf167e97b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.325050 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.325014    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpjs\" (UniqueName: \"kubernetes.io/projected/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-kube-api-access-8kpjs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.326322 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.326301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6blnx\" (UniqueName: \"kubernetes.io/projected/1cb09db2-2a8a-48a2-8270-ac314de8cc96-kube-api-access-6blnx\") pod \"node-ca-v7nh6\" (UID: \"1cb09db2-2a8a-48a2-8270-ac314de8cc96\") " pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.326599 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.326546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx8f5\" (UniqueName: \"kubernetes.io/projected/81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3-kube-api-access-mx8f5\") pod \"tuned-z7txb\" (UID: \"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3\") " pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.416324 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-var-lib-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-etc-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-config\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-var-lib-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovn-node-metrics-cert\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s54j\" (UniqueName: \"kubernetes.io/projected/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-kube-api-access-4s54j\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416508 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26xtg\" (UniqueName: \"kubernetes.io/projected/b6a2a639-00a1-46a0-a895-bb7f6c89c275-kube-api-access-26xtg\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-etc-openvswitch\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-kubelet\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-node-log\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-kubelet\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6p6h\" (UniqueName: \"kubernetes.io/projected/0bdb496b-80af-482b-bb44-661de8ec606c-kube-api-access-w6p6h\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a2a639-00a1-46a0-a895-bb7f6c89c275-tmp-dir\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-netns\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-ovn\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.416890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-node-log\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-cnibin\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-systemd-units\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-ovn\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-systemd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-systemd-units\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.416993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-netd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-bin\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6a2a639-00a1-46a0-a895-bb7f6c89c275-hosts-file\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-log-socket\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a2a639-00a1-46a0-a895-bb7f6c89c275-tmp-dir\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-system-cni-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-system-cni-dir\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-run-netns\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-bin\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-run-systemd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-log-socket\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.417637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bdb496b-80af-482b-bb44-661de8ec606c-cnibin\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6a2a639-00a1-46a0-a895-bb7f6c89c275-hosts-file\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-cni-netd\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-config\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-env-overrides\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-script-lib\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-slash\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-host-slash\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-env-overrides\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417808    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0bdb496b-80af-482b-bb44-661de8ec606c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.418311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.417909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovnkube-script-lib\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.419444 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.419422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-ovn-node-metrics-cert\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.427223 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.427186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s54j\" (UniqueName: \"kubernetes.io/projected/55a9eb53-a0ca-46a4-bcad-37c5deb99e3b-kube-api-access-4s54j\") pod \"ovnkube-node-lnfvq\" (UID: \"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.427354 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.427334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6p6h\" (UniqueName: \"kubernetes.io/projected/0bdb496b-80af-482b-bb44-661de8ec606c-kube-api-access-w6p6h\") pod \"multus-additional-cni-plugins-xzzp9\" (UID: \"0bdb496b-80af-482b-bb44-661de8ec606c\") " pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.427448 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.427429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26xtg\" (UniqueName: \"kubernetes.io/projected/b6a2a639-00a1-46a0-a895-bb7f6c89c275-kube-api-access-26xtg\") pod \"node-resolver-rhqrq\" (UID: \"b6a2a639-00a1-46a0-a895-bb7f6c89c275\") " pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.492350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.492269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k76fx"
Sep 16 15:50:05.501627 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.501606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v7nh6"
Sep 16 15:50:05.511206 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.511181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:05.515963 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.515941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws"
Sep 16 15:50:05.522729 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.522710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z7txb"
Sep 16 15:50:05.529431 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.529406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x4nbv"
Sep 16 15:50:05.537924 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.537906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rhqrq"
Sep 16 15:50:05.542575 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.542557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:05.548209 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.548183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzzp9"
Sep 16 15:50:05.656990 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.656830    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a2a639_00a1_46a0_a895_bb7f6c89c275.slice/crio-973a014c6c27301201dd94512460f191d6f5552eb1bc3d6056dae4552fcd12da WatchSource:0}: Error finding container 973a014c6c27301201dd94512460f191d6f5552eb1bc3d6056dae4552fcd12da: Status 404 returned error can't find the container with id 973a014c6c27301201dd94512460f191d6f5552eb1bc3d6056dae4552fcd12da
Sep 16 15:50:05.657605 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.657578    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdb496b_80af_482b_bb44_661de8ec606c.slice/crio-7465d411d84c9f380999491765234e379cac8415ec9695d857d8781efcaa309a WatchSource:0}: Error finding container 7465d411d84c9f380999491765234e379cac8415ec9695d857d8781efcaa309a: Status 404 returned error can't find the container with id 7465d411d84c9f380999491765234e379cac8415ec9695d857d8781efcaa309a
Sep 16 15:50:05.658936 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.658915    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda760ae3f_642b_4210_b55b_6809037b13fa.slice/crio-a9e6014591d8313347e6a57b44185c202db496507bc584cfa619fbc60a4a82a3 WatchSource:0}: Error finding container a9e6014591d8313347e6a57b44185c202db496507bc584cfa619fbc60a4a82a3: Status 404 returned error can't find the container with id a9e6014591d8313347e6a57b44185c202db496507bc584cfa619fbc60a4a82a3
Sep 16 15:50:05.660646 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.660330    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a9eb53_a0ca_46a4_bcad_37c5deb99e3b.slice/crio-0c952708c05dd6e44922abc163ffff6b1e0b84539f6f013f7347da443c670664 WatchSource:0}: Error finding container 0c952708c05dd6e44922abc163ffff6b1e0b84539f6f013f7347da443c670664: Status 404 returned error can't find the container with id 0c952708c05dd6e44922abc163ffff6b1e0b84539f6f013f7347da443c670664
Sep 16 15:50:05.663668 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.663651    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4621b60a_f864_49f5_bec2_0b28d89a9e89.slice/crio-8db0a09270e47fd3d427e16c1eefdd003d6baf5a5dcf17668f2d71df06504b72 WatchSource:0}: Error finding container 8db0a09270e47fd3d427e16c1eefdd003d6baf5a5dcf17668f2d71df06504b72: Status 404 returned error can't find the container with id 8db0a09270e47fd3d427e16c1eefdd003d6baf5a5dcf17668f2d71df06504b72
Sep 16 15:50:05.685884 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.685861    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81534bd9_c0ab_4e9b_80e5_81e11e9ee1b3.slice/crio-2777c999d5b0cd1f9f0b33001c3cb1c988e855dab9d4bb2cb134b9cf265bea66 WatchSource:0}: Error finding container 2777c999d5b0cd1f9f0b33001c3cb1c988e855dab9d4bb2cb134b9cf265bea66: Status 404 returned error can't find the container with id 2777c999d5b0cd1f9f0b33001c3cb1c988e855dab9d4bb2cb134b9cf265bea66
Sep 16 15:50:05.686563 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.686539    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8369695_3f94_44ef_8f93_cf5bf167e97b.slice/crio-776d8d5d5a21ca5ab37b7e136f1e0e7afbd1695c45a7bc662c8e29328f345c8f WatchSource:0}: Error finding container 776d8d5d5a21ca5ab37b7e136f1e0e7afbd1695c45a7bc662c8e29328f345c8f: Status 404 returned error can't find the container with id 776d8d5d5a21ca5ab37b7e136f1e0e7afbd1695c45a7bc662c8e29328f345c8f
Sep 16 15:50:05.687436 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:05.687411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb09db2_2a8a_48a2_8270_ac314de8cc96.slice/crio-d8ffb908af42d5ff886dd8933e6c04b350976cc4c7b55e4d368a68e8f1c2694c WatchSource:0}: Error finding container d8ffb908af42d5ff886dd8933e6c04b350976cc4c7b55e4d368a68e8f1c2694c: Status 404 returned error can't find the container with id d8ffb908af42d5ff886dd8933e6c04b350976cc4c7b55e4d368a68e8f1c2694c
Sep 16 15:50:05.820544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.820472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:05.820544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.820526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:05.820703 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.820610    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:05.820703 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.820614    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:05.820703 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.820658    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:06.820645475 +0000 UTC m=+4.048103458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:05.820703 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.820672    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:06.820665252 +0000 UTC m=+4.048123236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:05.921198 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:05.921139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:05.921336 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.921295    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:05.921336 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.921321    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:05.921336 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.921334    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:05.921509 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:05.921391    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:06.921377216 +0000 UTC m=+4.148835201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:06.239303 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.239128    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 15:45:05 +0000 UTC" deadline="2028-06-06 01:43:07.282121046 +0000 UTC"
Sep 16 15:50:06.239303 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.239201    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15081h53m1.042924543s"
Sep 16 15:50:06.316382 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.316294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v7nh6" event={"ID":"1cb09db2-2a8a-48a2-8270-ac314de8cc96","Type":"ContainerStarted","Data":"d8ffb908af42d5ff886dd8933e6c04b350976cc4c7b55e4d368a68e8f1c2694c"}
Sep 16 15:50:06.330833 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.330794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws" event={"ID":"f8369695-3f94-44ef-8f93-cf5bf167e97b","Type":"ContainerStarted","Data":"776d8d5d5a21ca5ab37b7e136f1e0e7afbd1695c45a7bc662c8e29328f345c8f"}
Sep 16 15:50:06.335678 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.335640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z7txb" event={"ID":"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3","Type":"ContainerStarted","Data":"2777c999d5b0cd1f9f0b33001c3cb1c988e855dab9d4bb2cb134b9cf265bea66"}
Sep 16 15:50:06.340553 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.340522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x4nbv" event={"ID":"4621b60a-f864-49f5-bec2-0b28d89a9e89","Type":"ContainerStarted","Data":"8db0a09270e47fd3d427e16c1eefdd003d6baf5a5dcf17668f2d71df06504b72"}
Sep 16 15:50:06.343803 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.343776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"0c952708c05dd6e44922abc163ffff6b1e0b84539f6f013f7347da443c670664"}
Sep 16 15:50:06.346092 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.346064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerStarted","Data":"7465d411d84c9f380999491765234e379cac8415ec9695d857d8781efcaa309a"}
Sep 16 15:50:06.355359 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.355330    2581 generic.go:356] "Generic (PLEG): container finished" podID="07b3d42f42d8f0ded147a5353becd15c" containerID="e33cef3e287508396245833463ea30f52b049614a722450ee64e85f58b711c8b" exitCode=0
Sep 16 15:50:06.355643 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.355591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal" event={"ID":"07b3d42f42d8f0ded147a5353becd15c","Type":"ContainerDied","Data":"e33cef3e287508396245833463ea30f52b049614a722450ee64e85f58b711c8b"}
Sep 16 15:50:06.359978 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.359954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k76fx" event={"ID":"f32d48d7-ec36-4a76-8782-071398693244","Type":"ContainerStarted","Data":"310b2f3ce1625398947c2cde5cc56e0e2c948bb57319cd78197eb72c407ba75e"}
Sep 16 15:50:06.366261 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.366224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krlk5" event={"ID":"a760ae3f-642b-4210-b55b-6809037b13fa","Type":"ContainerStarted","Data":"a9e6014591d8313347e6a57b44185c202db496507bc584cfa619fbc60a4a82a3"}
Sep 16 15:50:06.371675 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.371620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rhqrq" event={"ID":"b6a2a639-00a1-46a0-a895-bb7f6c89c275","Type":"ContainerStarted","Data":"973a014c6c27301201dd94512460f191d6f5552eb1bc3d6056dae4552fcd12da"}
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.829364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.829469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.829636    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.829699    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:08.829682012 +0000 UTC m=+6.057139998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.830112    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:06.830273 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.830182    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:08.830150953 +0000 UTC m=+6.057608940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:06.930913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:06.930309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:06.930913 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.930492    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:06.930913 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.930517    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:06.930913 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.930529    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:06.930913 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:06.930586    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:08.930567995 +0000 UTC m=+6.158025979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:07.239652 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.239564    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 15:45:05 +0000 UTC" deadline="2028-06-03 07:15:15.561594044 +0000 UTC"
Sep 16 15:50:07.239652 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.239600    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15015h25m8.321997656s"
Sep 16 15:50:07.304274 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.303753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:07.304274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:07.303885    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:07.304274 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.304261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:07.304573 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:07.304358    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:07.304573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.304537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:07.304671 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:07.304638    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:07.394754 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:07.394053    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal" event={"ID":"07b3d42f42d8f0ded147a5353becd15c","Type":"ContainerStarted","Data":"4df0a958a65bfa31f6f082afbce08b5bb84a2f93f43b40ce7e0ac72e5c0b73ce"}
Sep 16 15:50:08.846483 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:08.846438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:08.847066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:08.846537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:08.847066 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.846664    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:08.847066 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.846728    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:12.84670956 +0000 UTC m=+10.074167548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:08.847265 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.847149    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:08.847265 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.847222    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:12.847205484 +0000 UTC m=+10.074663467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:08.947796 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:08.947701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:08.947976 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.947852    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:08.947976 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.947884    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:08.947976 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.947897    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:08.947976 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:08.947967    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:12.947947357 +0000 UTC m=+10.175405364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:09.304396    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:09.304396    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:09.304522    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:09.304517    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:09.304593    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:09.304888 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:09.304654    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:11.307965 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:11.307926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:11.308477 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:11.308062    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:11.308477 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:11.308442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:11.308622 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:11.308544    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:11.308622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:11.308601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:11.308731 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:11.308683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:12.882013 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:12.881980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:12.882472 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:12.882050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:12.882472 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.882191    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:12.882472 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.882257    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:20.882238101 +0000 UTC m=+18.109696103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:12.882472 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.882188    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:12.882472 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.882358    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:20.882335619 +0000 UTC m=+18.109793614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:12.983311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:12.982710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:12.983311 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.982886    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:12.983311 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.982907    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:12.983311 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.982919    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:12.983311 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:12.982976    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:20.982959594 +0000 UTC m=+18.210417593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:13.304849 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:13.304663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:13.304849 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:13.304776    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:13.304849 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:13.304820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:13.305123 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:13.304881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:13.305123 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:13.304972    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:13.305123 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:13.305062    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:15.304333 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:15.304294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:15.304798 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:15.304305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:15.304798 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:15.304452    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:15.304798 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:15.304305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:15.304798 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:15.304554    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:15.304798 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:15.304659    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:17.304079 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:17.304041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:17.304079 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:17.304088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:17.304607 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:17.304041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:17.304607 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:17.304228    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:17.304607 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:17.304351    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:17.304607 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:17.304447    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:19.304549 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:19.304512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:19.304962 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:19.304512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:19.304962 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:19.304649    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:19.304962 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:19.304512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:19.304962 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:19.304693    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:19.304962 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:19.304810    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:20.937754 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:20.937712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:20.938274 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:20.937824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:20.938274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:20.937909    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:20.938274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:20.937940    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:20.938274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:20.937987    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:36.937970347 +0000 UTC m=+34.165428345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:20.938274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:20.938003    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:36.937994777 +0000 UTC m=+34.165452761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:21.038228 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:21.038177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:21.038397 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.038306    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:21.038397 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.038325    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:21.038397 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.038337    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:21.038397 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.038393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:50:37.038379185 +0000 UTC m=+34.265837173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:21.306953 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:21.306858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:21.306953 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:21.306858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:21.307142 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.306971    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:21.307142 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:21.306858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:21.307142 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.307090    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:21.307322 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:21.307182    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:23.305687 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:23.305645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:23.306122 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:23.305739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:23.306122 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:23.305773    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:23.306122 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:23.305817    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:23.306122 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:23.305873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:23.306122 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:23.305986    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:24.438074 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.437806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krlk5" event={"ID":"a760ae3f-642b-4210-b55b-6809037b13fa","Type":"ContainerStarted","Data":"99183fbe13ca896a1eb794fad633279b77e9ff5e2ed64af8bf605c8db9606e37"}
Sep 16 15:50:24.439310 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.439285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal" event={"ID":"6b7775f07a812f020e26327f5b0a385a","Type":"ContainerStarted","Data":"c8805b185d632fb7a739a887ebf8695b32584ba8261af623ed82c0d0a4b027c2"}
Sep 16 15:50:24.440610 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.440589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v7nh6" event={"ID":"1cb09db2-2a8a-48a2-8270-ac314de8cc96","Type":"ContainerStarted","Data":"c8410b15ddad0d8ab0d26a17a912387327c654e393b750ce8d88d62babeb6a9b"}
Sep 16 15:50:24.442066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.442043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws" event={"ID":"f8369695-3f94-44ef-8f93-cf5bf167e97b","Type":"ContainerStarted","Data":"6fb13050f8b26d46ff6fd9dc32a3121102f30d7b8b52dfc2bad1f26675609193"}
Sep 16 15:50:24.443401 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.443382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z7txb" event={"ID":"81534bd9-c0ab-4e9b-80e5-81e11e9ee1b3","Type":"ContainerStarted","Data":"b8f6b994f8d33753a181f42c5c34921b93114085d053f6dae4f896d1cf26df9f"}
Sep 16 15:50:24.454074 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.454038    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-krlk5" podStartSLOduration=2.934632351 podStartE2EDuration="21.454026574s" totalImagesPullingTime="18.519394223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:24.453994169 +0000 UTC m=+21.681452175" watchObservedRunningTime="2026-09-16 15:50:24.454026574 +0000 UTC m=+21.681484580"
Sep 16 15:50:24.454214 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.454188    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-111.ec2.internal" podStartSLOduration=20.454183363 podStartE2EDuration="20.454183363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:07.412095351 +0000 UTC m=+4.639553360" watchObservedRunningTime="2026-09-16 15:50:24.454183363 +0000 UTC m=+21.681641365"
Sep 16 15:50:24.484383 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.484334    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v7nh6" podStartSLOduration=2.995251266 podStartE2EDuration="21.4843092s" totalImagesPullingTime="18.489057934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:24.468046587 +0000 UTC m=+21.695504589" watchObservedRunningTime="2026-09-16 15:50:24.4843092 +0000 UTC m=+21.711767208"
Sep 16 15:50:24.500282 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.499739    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-111.ec2.internal" podStartSLOduration=20.499724461 podStartE2EDuration="20.499724461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:24.499030063 +0000 UTC m=+21.726488070" watchObservedRunningTime="2026-09-16 15:50:24.499724461 +0000 UTC m=+21.727182471"
Sep 16 15:50:24.500282 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:24.499816    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z7txb" podStartSLOduration=2.973912284 podStartE2EDuration="21.49981161s" totalImagesPullingTime="18.525899326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:24.484403884 +0000 UTC m=+21.711861895" watchObservedRunningTime="2026-09-16 15:50:24.49981161 +0000 UTC m=+21.727269616"
Sep 16 15:50:25.304499 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.304309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:25.304705 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.304313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:25.304705 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:25.304583    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:25.304705 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.304345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:25.304866 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:25.304650    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:25.304866 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:25.304753    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:25.447286 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.447247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rhqrq" event={"ID":"b6a2a639-00a1-46a0-a895-bb7f6c89c275","Type":"ContainerStarted","Data":"be68c26a2a472e5c1e48b2a5b1bc164f9a924626ea5bcdc5f8a9f9265dad26aa"}
Sep 16 15:50:25.448827 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.448805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x4nbv" event={"ID":"4621b60a-f864-49f5-bec2-0b28d89a9e89","Type":"ContainerStarted","Data":"63cb0b26fda7a15fdd3f42fbf2ee045d4bd29ba6e6ea7c02ec7be096fa7d461d"}
Sep 16 15:50:25.451477 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451459    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:50:25.451763 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451746    2581 generic.go:356] "Generic (PLEG): container finished" podID="55a9eb53-a0ca-46a4-bcad-37c5deb99e3b" containerID="4d3020fe1bdd75a073b1387d81a29429f066ee9a0a0a50ec0450c8cb56687f10" exitCode=1
Sep 16 15:50:25.451826 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"dd3b2fbab7274445efe0821d9f2aee4c61dde461a022fa5b0eebace2d140c0f6"}
Sep 16 15:50:25.451880 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451861    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"298c9a3e0dc599d7f6f24dd3587a93885f43985f47018598474cecfc8dfefc64"}
Sep 16 15:50:25.451936 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"a16e9b006b739d7eb796892dfede1f3b8449c88a8b096194d83b56f4bcc16cdb"}
Sep 16 15:50:25.451936 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"593bae6fb7f2df2dda9da26ca0f7de8ac928c6ccc9b408bbd13b44f845e085b1"}
Sep 16 15:50:25.452021 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451956    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerDied","Data":"4d3020fe1bdd75a073b1387d81a29429f066ee9a0a0a50ec0450c8cb56687f10"}
Sep 16 15:50:25.452021 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.451984    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"25e2a617f08cfc621fdff37e9bd06f1c834a281b15600383be66d611dbaa615e"}
Sep 16 15:50:25.453241 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.453221    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="95dd10ba117a1825d0b0ecf9c7de3ddb790f51719178f578110f155713aba0e0" exitCode=0
Sep 16 15:50:25.453331 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.453305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"95dd10ba117a1825d0b0ecf9c7de3ddb790f51719178f578110f155713aba0e0"}
Sep 16 15:50:25.467181 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.467131    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rhqrq" podStartSLOduration=3.900839023 podStartE2EDuration="22.467118576s" totalImagesPullingTime="18.566279553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:25.466901181 +0000 UTC m=+22.694359187" watchObservedRunningTime="2026-09-16 15:50:25.467118576 +0000 UTC m=+22.694576583"
Sep 16 15:50:25.487745 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.487705    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x4nbv" podStartSLOduration=3.232266296 podStartE2EDuration="22.487687851s" totalImagesPullingTime="19.255421555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:25.487212931 +0000 UTC m=+22.714670941" watchObservedRunningTime="2026-09-16 15:50:25.487687851 +0000 UTC m=+22.715145858"
Sep 16 15:50:25.583903 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:25.583781    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 15:50:26.273398 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.273287    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T15:50:25.583898125Z","UUID":"2c8a41f8-d311-4303-a93b-01aa6f8a3da5","Handler":null,"Name":"","Endpoint":""}
Sep 16 15:50:26.275136 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.275111    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 15:50:26.275287 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.275148    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 15:50:26.297266 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.297230    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:26.457753 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.457710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k76fx" event={"ID":"f32d48d7-ec36-4a76-8782-071398693244","Type":"ContainerStarted","Data":"a884040924365bb8a13de3ddde4a3c7bf34b64b10093e852a66a6441361c94b9"}
Sep 16 15:50:26.459836 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.459808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws" event={"ID":"f8369695-3f94-44ef-8f93-cf5bf167e97b","Type":"ContainerStarted","Data":"260b59e78931e6b21f45087671f219e9076f3ccb0c106012eb6754e634ff1155"}
Sep 16 15:50:26.473039 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:26.472999    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k76fx" podStartSLOduration=4.948787389 podStartE2EDuration="23.472986103s" totalImagesPullingTime="18.524198714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:26.472480325 +0000 UTC m=+23.699938332" watchObservedRunningTime="2026-09-16 15:50:26.472986103 +0000 UTC m=+23.700444110"
Sep 16 15:50:27.304187 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.304086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:27.304187 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.304125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:27.304410 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:27.304228    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:27.304410 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:27.304327    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:27.304410 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.304392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:27.304572 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:27.304460    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:27.465755 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.465711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws" event={"ID":"f8369695-3f94-44ef-8f93-cf5bf167e97b","Type":"ContainerStarted","Data":"957d57dd9218b5f5bd7cfffab0aa80a34caf294ff39605d291487456e68e5c34"}
Sep 16 15:50:27.469406 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.469384    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:50:27.469926 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.469901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"d03206ca17d87c8a0db2cd1dd655a619addfded0f9f4d521c82a20179b1e395f"}
Sep 16 15:50:27.484095 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:27.484045    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlzws" podStartSLOduration=3.989384079 podStartE2EDuration="24.484033274s" totalImagesPullingTime="20.494649195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:27.483485373 +0000 UTC m=+24.710943370" watchObservedRunningTime="2026-09-16 15:50:27.484033274 +0000 UTC m=+24.711491280"
Sep 16 15:50:28.491940 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:28.491905    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:28.492554 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:28.492520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:29.304039 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:29.304002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:29.304240 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:29.304003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:29.304240 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:29.304135    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:29.304240 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:29.304127    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:29.304430 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:29.304247    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:29.304430 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:29.304343    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:29.478533 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:29.478508    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:50:29.479584 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:29.479565    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-krlk5"
Sep 16 15:50:30.484353 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.484179    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:50:30.484768 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.484695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"58f8b1f4a5effb87a2cb485d50fb71f095e0f4db80ec70209bb2686261340f38"}
Sep 16 15:50:30.485031 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.485001    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:30.485192 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.485174    2581 scope.go:117] "RemoveContainer" containerID="4d3020fe1bdd75a073b1387d81a29429f066ee9a0a0a50ec0450c8cb56687f10"
Sep 16 15:50:30.486792 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.486768    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="b7b31a854ff76518b26e6f0131bd650ac726793f9df2ccd8b4d67494fb8d330a" exitCode=0
Sep 16 15:50:30.486889 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.486851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"b7b31a854ff76518b26e6f0131bd650ac726793f9df2ccd8b4d67494fb8d330a"}
Sep 16 15:50:30.501629 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:30.501610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:31.303905 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.303872    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:31.304031 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.303915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:31.304031 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.303884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:31.304031 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.304012    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:31.304227 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.304061    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:31.304227 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.304126    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:31.338084 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.338052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rv47s"]
Sep 16 15:50:31.341362 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.341335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hsntg"]
Sep 16 15:50:31.342017 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.341993    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2xlht"]
Sep 16 15:50:31.492703 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.492677    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:50:31.493657 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.493052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" event={"ID":"55a9eb53-a0ca-46a4-bcad-37c5deb99e3b","Type":"ContainerStarted","Data":"b53ffbcaf168bfba2e847e96ae9c997d2c142042348193ec42cdfbd35798fce1"}
Sep 16 15:50:31.493657 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.493267    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:31.493657 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.493338    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:31.495446 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.495420    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="d3a4c47ece8681e5d6a939a318e4a33544a242d107fa3ecb97d0eaf7eebe17b2" exitCode=0
Sep 16 15:50:31.495565 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.495496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:31.495565 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.495528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"d3a4c47ece8681e5d6a939a318e4a33544a242d107fa3ecb97d0eaf7eebe17b2"}
Sep 16 15:50:31.495668 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.495641    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:31.495714 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.495680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:31.495782 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.495765    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:31.495829 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.495810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:31.495877 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:31.495865    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:31.508742 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.508724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:50:31.521921 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:31.521886    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq" podStartSLOduration=9.952031651 podStartE2EDuration="28.521874717s" totalImagesPullingTime="18.569843066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:31.520455811 +0000 UTC m=+28.747913819" watchObservedRunningTime="2026-09-16 15:50:31.521874717 +0000 UTC m=+28.749332716"
Sep 16 15:50:33.307354 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:33.307116    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:33.307740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:33.307206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:33.307740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:33.307239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:33.307740 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:33.307488    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:33.307740 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:33.307529    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:33.307740 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:33.307598    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:33.503087 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:33.502981    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="6e826deaccb18b8922bbcc5216d6b60619e7d4c1d625565b95369086ec0aa363" exitCode=0
Sep 16 15:50:33.503087 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:33.503070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"6e826deaccb18b8922bbcc5216d6b60619e7d4c1d625565b95369086ec0aa363"}
Sep 16 15:50:35.304293 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:35.304259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:35.304777 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:35.304376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:35.304777 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:35.304388    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:35.304777 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:35.304474    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:35.304777 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:35.304524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:35.304777 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:35.304579    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:36.945875 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:36.945831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:36.946429 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:36.945905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:36.946429 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:36.946000    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:36.946429 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:36.946047    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:36.946429 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:36.946069    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret podName:08ec670f-7702-4bae-a7c9-2e94c3b511b5 nodeName:}" failed. No retries permitted until 2026-09-16 15:51:08.946053898 +0000 UTC m=+66.173511901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret") pod "global-pull-secret-syncer-hsntg" (UID: "08ec670f-7702-4bae-a7c9-2e94c3b511b5") : object "kube-system"/"original-pull-secret" not registered
Sep 16 15:50:36.946429 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:36.946101    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:51:08.946083158 +0000 UTC m=+66.173541175 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 15:50:37.047341 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.047305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:37.047509 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.047471    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 15:50:37.047509 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.047503    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 15:50:37.047594 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.047514    2581 projected.go:196] Error preparing data for projected volume kube-api-access-zpqrm for pod openshift-network-diagnostics/network-check-target-2xlht: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:37.047594 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.047569    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm podName:158b54bc-8323-4243-bc25-e96ea23ad692 nodeName:}" failed. No retries permitted until 2026-09-16 15:51:09.047554003 +0000 UTC m=+66.275012002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zpqrm" (UniqueName: "kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm") pod "network-check-target-2xlht" (UID: "158b54bc-8323-4243-bc25-e96ea23ad692") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 15:50:37.304875 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.304845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:37.305027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.304845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:37.305027 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.304845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:37.305027 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.304990    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:50:37.305214 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.305072    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsntg" podUID="08ec670f-7702-4bae-a7c9-2e94c3b511b5"
Sep 16 15:50:37.305214 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.305172    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2xlht" podUID="158b54bc-8323-4243-bc25-e96ea23ad692"
Sep 16 15:50:37.558171 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.558087    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-111.ec2.internal" event="NodeReady"
Sep 16 15:50:37.558306 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.558211    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 15:50:37.606388 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.606360    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vdwrr"]
Sep 16 15:50:37.620101 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.620072    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hbb4s"]
Sep 16 15:50:37.620254 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.620202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.624568 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.624542    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-thq8g\""
Sep 16 15:50:37.624847 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.624824    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 15:50:37.625188 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.625172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 15:50:37.634117 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.634096    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdwrr"]
Sep 16 15:50:37.634225 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.634138    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hbb4s"]
Sep 16 15:50:37.634267 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.634220    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.636882 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.636860    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 15:50:37.637022 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.636957    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j6n8k\""
Sep 16 15:50:37.637237 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.637218    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 15:50:37.637464 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.637452    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 15:50:37.752974 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.752935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39025b22-64ea-4d4b-a568-69ccf39c1e3f-tmp-dir\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.752974 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.752969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22113a5f-d398-4e16-b11e-b96c75d5e2e9-cert\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.753204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.753042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbjls\" (UniqueName: \"kubernetes.io/projected/39025b22-64ea-4d4b-a568-69ccf39c1e3f-kube-api-access-gbjls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.753204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.753060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcgr7\" (UniqueName: \"kubernetes.io/projected/22113a5f-d398-4e16-b11e-b96c75d5e2e9-kube-api-access-bcgr7\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.753204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.753085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.753204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.753101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39025b22-64ea-4d4b-a568-69ccf39c1e3f-config-volume\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.853801 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39025b22-64ea-4d4b-a568-69ccf39c1e3f-tmp-dir\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.853801 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22113a5f-d398-4e16-b11e-b96c75d5e2e9-cert\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.854024 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbjls\" (UniqueName: \"kubernetes.io/projected/39025b22-64ea-4d4b-a568-69ccf39c1e3f-kube-api-access-gbjls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.854024 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcgr7\" (UniqueName: \"kubernetes.io/projected/22113a5f-d398-4e16-b11e-b96c75d5e2e9-kube-api-access-bcgr7\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.854024 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.854024 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.853896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39025b22-64ea-4d4b-a568-69ccf39c1e3f-config-volume\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.854024 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.853996    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:37.854298 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:37.854067    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:50:38.354046561 +0000 UTC m=+35.581504561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:50:37.854298 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.854095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39025b22-64ea-4d4b-a568-69ccf39c1e3f-tmp-dir\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.854544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.854525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39025b22-64ea-4d4b-a568-69ccf39c1e3f-config-volume\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.858604 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.858576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22113a5f-d398-4e16-b11e-b96c75d5e2e9-cert\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.881079 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.881059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbjls\" (UniqueName: \"kubernetes.io/projected/39025b22-64ea-4d4b-a568-69ccf39c1e3f-kube-api-access-gbjls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:37.881764 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.881211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcgr7\" (UniqueName: \"kubernetes.io/projected/22113a5f-d398-4e16-b11e-b96c75d5e2e9-kube-api-access-bcgr7\") pod \"ingress-canary-hbb4s\" (UID: \"22113a5f-d398-4e16-b11e-b96c75d5e2e9\") " pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:37.943222 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:37.943192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hbb4s"
Sep 16 15:50:38.268297 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:38.268261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hbb4s"]
Sep 16 15:50:38.272362 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:50:38.272327    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22113a5f_d398_4e16_b11e_b96c75d5e2e9.slice/crio-00b43f5dc43b6225d34f28ee95d40a04ddfe059e950d5fb84db5c0c1ab8d6f51 WatchSource:0}: Error finding container 00b43f5dc43b6225d34f28ee95d40a04ddfe059e950d5fb84db5c0c1ab8d6f51: Status 404 returned error can't find the container with id 00b43f5dc43b6225d34f28ee95d40a04ddfe059e950d5fb84db5c0c1ab8d6f51
Sep 16 15:50:38.358050 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:38.358012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:38.358228 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:38.358186    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:38.358273 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:38.358257    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:50:39.358241213 +0000 UTC m=+36.585699197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:50:38.519805 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:38.519735    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="2a085bbaa22afb8621309b8af047487c5ef18c7bdfc6d42e0f4e347f9557ab39" exitCode=0
Sep 16 15:50:38.519958 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:38.519821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"2a085bbaa22afb8621309b8af047487c5ef18c7bdfc6d42e0f4e347f9557ab39"}
Sep 16 15:50:38.521019 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:38.521000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hbb4s" event={"ID":"22113a5f-d398-4e16-b11e-b96c75d5e2e9","Type":"ContainerStarted","Data":"00b43f5dc43b6225d34f28ee95d40a04ddfe059e950d5fb84db5c0c1ab8d6f51"}
Sep 16 15:50:39.304298 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.304256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:50:39.304706 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.304257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:50:39.304706 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.304399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:50:39.307341 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.307314    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 15:50:39.308392 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.308374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mmzn4\""
Sep 16 15:50:39.308489 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.308465    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hlg9r\""
Sep 16 15:50:39.308489 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.308473    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 15:50:39.308597 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.308505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 15:50:39.308597 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.308473    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 15:50:39.366122 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.366087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:39.366274 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:39.366251    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:39.366326 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:39.366315    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:50:41.366299978 +0000 UTC m=+38.593757962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:50:39.526667 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.526633    2581 generic.go:356] "Generic (PLEG): container finished" podID="0bdb496b-80af-482b-bb44-661de8ec606c" containerID="3083e83e037bb65cdaf915fa6a57a30bd8c7719cacedaa0ca0681b4597e3c6b5" exitCode=0
Sep 16 15:50:39.526667 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:39.526670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerDied","Data":"3083e83e037bb65cdaf915fa6a57a30bd8c7719cacedaa0ca0681b4597e3c6b5"}
Sep 16 15:50:40.534019 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:40.533980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" event={"ID":"0bdb496b-80af-482b-bb44-661de8ec606c","Type":"ContainerStarted","Data":"8d35c696b46093d2a44bdac3abe0cce5e7e11feb5a72a1cdceaf77a0ae764839"}
Sep 16 15:50:40.560567 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:40.560498    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzzp9" podStartSLOduration=8.228652141 podStartE2EDuration="37.560474887s" totalImagesPullingTime="29.331822746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:40.558406416 +0000 UTC m=+37.785864423" watchObservedRunningTime="2026-09-16 15:50:40.560474887 +0000 UTC m=+37.787932893"
Sep 16 15:50:41.384249 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:41.384206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:41.384436 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:41.384342    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:41.384436 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:41.384407    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:50:45.384389481 +0000 UTC m=+42.611847478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:50:42.540003 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:42.539811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hbb4s" event={"ID":"22113a5f-d398-4e16-b11e-b96c75d5e2e9","Type":"ContainerStarted","Data":"0980f0277847be5e3c4aaa0e57692d3872d56215bcda3f64336bd82e5a031207"}
Sep 16 15:50:42.556772 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:42.556726    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hbb4s" podStartSLOduration=1.84177551 podStartE2EDuration="5.556713288s" totalImagesPullingTime="3.714937778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:50:42.556402995 +0000 UTC m=+39.783861001" watchObservedRunningTime="2026-09-16 15:50:42.556713288 +0000 UTC m=+39.784171290"
Sep 16 15:50:45.410091 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:45.410050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:45.410578 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:45.410247    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:45.410578 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:45.410332    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:50:53.41031168 +0000 UTC m=+50.637769691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:50:53.464685 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:50:53.464645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:50:53.465264 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:53.464829    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:50:53.465264 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:50:53.464918    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:51:09.464895614 +0000 UTC m=+66.692353606 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:51:03.513530 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:03.513491    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lnfvq"
Sep 16 15:51:08.976631 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:08.976580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:51:08.977085 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:08.976645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:51:08.979669 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:08.979651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 15:51:08.979720 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:08.979652    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 15:51:08.987745 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:08.987726    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 15:51:08.987842 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:08.987809    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:12.98779272 +0000 UTC m=+130.215250704 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : secret "metrics-daemon-secret" not found
Sep 16 15:51:08.991102 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:08.991083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/08ec670f-7702-4bae-a7c9-2e94c3b511b5-original-pull-secret\") pod \"global-pull-secret-syncer-hsntg\" (UID: \"08ec670f-7702-4bae-a7c9-2e94c3b511b5\") " pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:51:09.024377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.024347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsntg"
Sep 16 15:51:09.077344 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.077314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:51:09.080410 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.080390    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 15:51:09.091007 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.090815    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 15:51:09.102119 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.102096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpqrm\" (UniqueName: \"kubernetes.io/projected/158b54bc-8323-4243-bc25-e96ea23ad692-kube-api-access-zpqrm\") pod \"network-check-target-2xlht\" (UID: \"158b54bc-8323-4243-bc25-e96ea23ad692\") " pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:51:09.316732 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.316664    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hlg9r\""
Sep 16 15:51:09.324095 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.324066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:51:09.324992 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.324969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hsntg"]
Sep 16 15:51:09.328131 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:51:09.328108    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ec670f_7702_4bae_a7c9_2e94c3b511b5.slice/crio-c50c388ecfe39e6fa56d0b164c182982c430e40d85b4461a5d2fb4e6ce4c92c4 WatchSource:0}: Error finding container c50c388ecfe39e6fa56d0b164c182982c430e40d85b4461a5d2fb4e6ce4c92c4: Status 404 returned error can't find the container with id c50c388ecfe39e6fa56d0b164c182982c430e40d85b4461a5d2fb4e6ce4c92c4
Sep 16 15:51:09.479943 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.479897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:51:09.480099 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:09.480054    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:51:09.480138 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:09.480125    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:51:41.480107758 +0000 UTC m=+98.707565748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:51:09.592583 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.592503    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2xlht"]
Sep 16 15:51:09.595600 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:51:09.595573    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158b54bc_8323_4243_bc25_e96ea23ad692.slice/crio-ccf4d90f24f9089f4a48e9dead3c3172649ab5cf03bccf87cccbe79ec610561f WatchSource:0}: Error finding container ccf4d90f24f9089f4a48e9dead3c3172649ab5cf03bccf87cccbe79ec610561f: Status 404 returned error can't find the container with id ccf4d90f24f9089f4a48e9dead3c3172649ab5cf03bccf87cccbe79ec610561f
Sep 16 15:51:09.607199 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.607152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2xlht" event={"ID":"158b54bc-8323-4243-bc25-e96ea23ad692","Type":"ContainerStarted","Data":"ccf4d90f24f9089f4a48e9dead3c3172649ab5cf03bccf87cccbe79ec610561f"}
Sep 16 15:51:09.608285 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:09.608265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hsntg" event={"ID":"08ec670f-7702-4bae-a7c9-2e94c3b511b5","Type":"ContainerStarted","Data":"c50c388ecfe39e6fa56d0b164c182982c430e40d85b4461a5d2fb4e6ce4c92c4"}
Sep 16 15:51:14.624599 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:14.624558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2xlht" event={"ID":"158b54bc-8323-4243-bc25-e96ea23ad692","Type":"ContainerStarted","Data":"ed3bfcef943fc89fcea0d9227383116d36ddac0cdd150f7ece9e6d82dd7f57ed"}
Sep 16 15:51:14.625028 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:14.624797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:51:14.626065 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:14.626044    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hsntg" event={"ID":"08ec670f-7702-4bae-a7c9-2e94c3b511b5","Type":"ContainerStarted","Data":"57f8a00d8794e659bcd0cff6c92aa265da7ca29c16328bd2a71fa014a58eb034"}
Sep 16 15:51:14.640686 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:14.640642    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2xlht" podStartSLOduration=67.446607699 podStartE2EDuration="1m11.640629193s" totalImagesPullingTime="4.194021494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:51:14.640046145 +0000 UTC m=+71.867504153" watchObservedRunningTime="2026-09-16 15:51:14.640629193 +0000 UTC m=+71.868087199"
Sep 16 15:51:14.656473 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:14.656426    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hsntg" podStartSLOduration=67.190889452 podStartE2EDuration="1m11.6564098s" totalImagesPullingTime="4.465520348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:51:14.655868236 +0000 UTC m=+71.883326245" watchObservedRunningTime="2026-09-16 15:51:14.6564098 +0000 UTC m=+71.883867807"
Sep 16 15:51:41.495587 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:41.495546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:51:41.496082 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:41.495716    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 15:51:41.496082 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:51:41.495814    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls podName:39025b22-64ea-4d4b-a568-69ccf39c1e3f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:45.495790273 +0000 UTC m=+162.723248271 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls") pod "dns-default-vdwrr" (UID: "39025b22-64ea-4d4b-a568-69ccf39c1e3f") : secret "dns-default-metrics-tls" not found
Sep 16 15:51:45.630501 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:51:45.630467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2xlht"
Sep 16 15:52:05.883118 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.883088    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"]
Sep 16 15:52:05.885785 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.885769    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:05.888568 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.888546    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 15:52:05.888568 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.888557    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:52:05.888843 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.888827    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 15:52:05.889179 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.889164    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rzltr\""
Sep 16 15:52:05.889514 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.889500    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 15:52:05.898455 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.898435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"]
Sep 16 15:52:05.952290 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.952252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d24r\" (UniqueName: \"kubernetes.io/projected/fa9bf1a4-1949-4346-bdb4-88795e371dd2-kube-api-access-8d24r\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:05.952459 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.952305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa9bf1a4-1949-4346-bdb4-88795e371dd2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:05.952459 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.952361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa9bf1a4-1949-4346-bdb4-88795e371dd2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:05.982375 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.982339    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"]
Sep 16 15:52:05.985262 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.985236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"
Sep 16 15:52:05.986337 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.986316    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-64ghg"]
Sep 16 15:52:05.988252 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.988234    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dd8nm\""
Sep 16 15:52:05.988348 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.988297    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:52:05.988702 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.988689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 15:52:05.988843 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.988827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:05.990046 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.990023    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:52:05.992710 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.992694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:05.993143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.993124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 15:52:05.993427 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.993408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qm5r9\""
Sep 16 15:52:05.993636 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.993621    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 15:52:05.993692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.993636    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 15:52:05.993692 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.993653    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:52:05.995456 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.995429    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 15:52:05.995932 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.995911    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"]
Sep 16 15:52:05.996431 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.996406    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 15:52:05.996527 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.996516    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c92pf\""
Sep 16 15:52:05.997311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:05.997290    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 15:52:06.009145 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.009120    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 15:52:06.010279 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.010263    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 15:52:06.016310 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.016290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:52:06.017292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.017269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-64ghg"]
Sep 16 15:52:06.053548 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnhn8\" (UniqueName: \"kubernetes.io/projected/70b27a36-718e-4e5c-bd85-a9a8844ae0ff-kube-api-access-rnhn8\") pod \"volume-data-source-validator-86695b7cc7-c257x\" (UID: \"70b27a36-718e-4e5c-bd85-a9a8844ae0ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"
Sep 16 15:52:06.053548 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-config\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.053757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.053757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.053757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c46801f9-26da-47f0-aedb-97c459b68ef8-serving-cert\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.053757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-trusted-ca\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.053757 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.053949 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.053949 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa9bf1a4-1949-4346-bdb4-88795e371dd2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.053949 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.053949 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbw9p\" (UniqueName: \"kubernetes.io/projected/c46801f9-26da-47f0-aedb-97c459b68ef8-kube-api-access-xbw9p\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.053949 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d24r\" (UniqueName: \"kubernetes.io/projected/fa9bf1a4-1949-4346-bdb4-88795e371dd2-kube-api-access-8d24r\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.054236 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.053976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.054236 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.054009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.054236 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.054043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpzm\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.054236 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.054088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa9bf1a4-1949-4346-bdb4-88795e371dd2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.054435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.054317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa9bf1a4-1949-4346-bdb4-88795e371dd2-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.056317 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.056295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa9bf1a4-1949-4346-bdb4-88795e371dd2-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.072384 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.072362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d24r\" (UniqueName: \"kubernetes.io/projected/fa9bf1a4-1949-4346-bdb4-88795e371dd2-kube-api-access-8d24r\") pod \"kube-storage-version-migrator-operator-6c8f84f897-rcphw\" (UID: \"fa9bf1a4-1949-4346-bdb4-88795e371dd2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.094001 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.093974    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"]
Sep 16 15:52:06.097049 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.097034    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.099597 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.099576    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 15:52:06.099691 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.099606    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 15:52:06.099752 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.099724    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 15:52:06.099957 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.099941    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zjdb4\""
Sep 16 15:52:06.107040 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.107020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"]
Sep 16 15:52:06.154868 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.154797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-config\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.154868 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.154829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.154868 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.154850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.154880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c46801f9-26da-47f0-aedb-97c459b68ef8-serving-cert\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.155066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.154903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-trusted-ca\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.155066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155233 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155233 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155233 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbw9p\" (UniqueName: \"kubernetes.io/projected/c46801f9-26da-47f0-aedb-97c459b68ef8-kube-api-access-xbw9p\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.155233 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpzm\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6kss\" (UniqueName: \"kubernetes.io/projected/107e8198-50ce-4b67-ad64-67f00fcf7e59-kube-api-access-w6kss\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.155411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.155411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnhn8\" (UniqueName: \"kubernetes.io/projected/70b27a36-718e-4e5c-bd85-a9a8844ae0ff-kube-api-access-rnhn8\") pod \"volume-data-source-validator-86695b7cc7-c257x\" (UID: \"70b27a36-718e-4e5c-bd85-a9a8844ae0ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"
Sep 16 15:52:06.155660 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155720 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-config\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.155776 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155829 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.155806    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 15:52:06.155829 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.155822    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cdf7fd65b-sdfwv: secret "image-registry-tls" not found
Sep 16 15:52:06.155925 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.155925 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.155895    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls podName:b04d6963-a011-47b3-add8-44ac3506806f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:06.655874152 +0000 UTC m=+123.883332182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls") pod "image-registry-cdf7fd65b-sdfwv" (UID: "b04d6963-a011-47b3-add8-44ac3506806f") : secret "image-registry-tls" not found
Sep 16 15:52:06.156038 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.155964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c46801f9-26da-47f0-aedb-97c459b68ef8-trusted-ca\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.157606 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.157581    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c46801f9-26da-47f0-aedb-97c459b68ef8-serving-cert\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.158037 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.158016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.158077 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.158055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.173136 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.173107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbw9p\" (UniqueName: \"kubernetes.io/projected/c46801f9-26da-47f0-aedb-97c459b68ef8-kube-api-access-xbw9p\") pod \"console-operator-696884fb9c-64ghg\" (UID: \"c46801f9-26da-47f0-aedb-97c459b68ef8\") " pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.173353 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.173335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.173412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.173362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnhn8\" (UniqueName: \"kubernetes.io/projected/70b27a36-718e-4e5c-bd85-a9a8844ae0ff-kube-api-access-rnhn8\") pod \"volume-data-source-validator-86695b7cc7-c257x\" (UID: \"70b27a36-718e-4e5c-bd85-a9a8844ae0ff\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"
Sep 16 15:52:06.173543 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.173523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpzm\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.194120 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.194096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"
Sep 16 15:52:06.255980 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.255936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6kss\" (UniqueName: \"kubernetes.io/projected/107e8198-50ce-4b67-ad64-67f00fcf7e59-kube-api-access-w6kss\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.256126 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.255995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.256126 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.256117    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 15:52:06.256265 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.256203    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls podName:107e8198-50ce-4b67-ad64-67f00fcf7e59 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:06.756182175 +0000 UTC m=+123.983640171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rvphr" (UID: "107e8198-50ce-4b67-ad64-67f00fcf7e59") : secret "samples-operator-tls" not found
Sep 16 15:52:06.264258 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.264233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6kss\" (UniqueName: \"kubernetes.io/projected/107e8198-50ce-4b67-ad64-67f00fcf7e59-kube-api-access-w6kss\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.296012 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.295984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"
Sep 16 15:52:06.303212 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.303191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:06.460627 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.460495    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw"]
Sep 16 15:52:06.462891 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:06.462866    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9bf1a4_1949_4346_bdb4_88795e371dd2.slice/crio-93286c402b7f1bd8a03f5aae6b1b634bf3ea461c119e33f42451a8a41ae9467d WatchSource:0}: Error finding container 93286c402b7f1bd8a03f5aae6b1b634bf3ea461c119e33f42451a8a41ae9467d: Status 404 returned error can't find the container with id 93286c402b7f1bd8a03f5aae6b1b634bf3ea461c119e33f42451a8a41ae9467d
Sep 16 15:52:06.566687 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.566658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x"]
Sep 16 15:52:06.569230 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:06.569202    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b27a36_718e_4e5c_bd85_a9a8844ae0ff.slice/crio-c53fb265ae0c4ca9bb3521c9c160245e2358ac7bdae62557e481c3673ed2a7a8 WatchSource:0}: Error finding container c53fb265ae0c4ca9bb3521c9c160245e2358ac7bdae62557e481c3673ed2a7a8: Status 404 returned error can't find the container with id c53fb265ae0c4ca9bb3521c9c160245e2358ac7bdae62557e481c3673ed2a7a8
Sep 16 15:52:06.571005 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.570985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-64ghg"]
Sep 16 15:52:06.573728 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:06.573706    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46801f9_26da_47f0_aedb_97c459b68ef8.slice/crio-63c6f19f57ee8589ba55786c95432498d24e464146b5ab0064d5b328a1976ff1 WatchSource:0}: Error finding container 63c6f19f57ee8589ba55786c95432498d24e464146b5ab0064d5b328a1976ff1: Status 404 returned error can't find the container with id 63c6f19f57ee8589ba55786c95432498d24e464146b5ab0064d5b328a1976ff1
Sep 16 15:52:06.658933 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.658895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:06.659095 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.659051    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 15:52:06.659095 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.659067    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cdf7fd65b-sdfwv: secret "image-registry-tls" not found
Sep 16 15:52:06.659203 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.659121    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls podName:b04d6963-a011-47b3-add8-44ac3506806f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:07.659103536 +0000 UTC m=+124.886561537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls") pod "image-registry-cdf7fd65b-sdfwv" (UID: "b04d6963-a011-47b3-add8-44ac3506806f") : secret "image-registry-tls" not found
Sep 16 15:52:06.751673 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.751589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" event={"ID":"c46801f9-26da-47f0-aedb-97c459b68ef8","Type":"ContainerStarted","Data":"63c6f19f57ee8589ba55786c95432498d24e464146b5ab0064d5b328a1976ff1"}
Sep 16 15:52:06.752634 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.752615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x" event={"ID":"70b27a36-718e-4e5c-bd85-a9a8844ae0ff","Type":"ContainerStarted","Data":"c53fb265ae0c4ca9bb3521c9c160245e2358ac7bdae62557e481c3673ed2a7a8"}
Sep 16 15:52:06.753581 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.753560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw" event={"ID":"fa9bf1a4-1949-4346-bdb4-88795e371dd2","Type":"ContainerStarted","Data":"93286c402b7f1bd8a03f5aae6b1b634bf3ea461c119e33f42451a8a41ae9467d"}
Sep 16 15:52:06.759927 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:06.759908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:06.760022 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.760011    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 15:52:06.760064 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:06.760061    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls podName:107e8198-50ce-4b67-ad64-67f00fcf7e59 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:07.760048497 +0000 UTC m=+124.987506480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rvphr" (UID: "107e8198-50ce-4b67-ad64-67f00fcf7e59") : secret "samples-operator-tls" not found
Sep 16 15:52:07.667778 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:07.667734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:07.668320 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:07.667875    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 15:52:07.668320 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:07.667894    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cdf7fd65b-sdfwv: secret "image-registry-tls" not found
Sep 16 15:52:07.668320 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:07.667961    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls podName:b04d6963-a011-47b3-add8-44ac3506806f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:09.667940515 +0000 UTC m=+126.895398514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls") pod "image-registry-cdf7fd65b-sdfwv" (UID: "b04d6963-a011-47b3-add8-44ac3506806f") : secret "image-registry-tls" not found
Sep 16 15:52:07.768307 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:07.768278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:07.768426 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:07.768407    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 15:52:07.768476 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:07.768471    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls podName:107e8198-50ce-4b67-ad64-67f00fcf7e59 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:09.768456036 +0000 UTC m=+126.995914020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rvphr" (UID: "107e8198-50ce-4b67-ad64-67f00fcf7e59") : secret "samples-operator-tls" not found
Sep 16 15:52:08.760120 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:08.760071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x" event={"ID":"70b27a36-718e-4e5c-bd85-a9a8844ae0ff","Type":"ContainerStarted","Data":"ef66e1d7e530bb2a41b634bf6666417d95ce6d32db40c6f941948ab7870f685e"}
Sep 16 15:52:08.777377 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:08.777315    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c257x" podStartSLOduration=2.601454706 podStartE2EDuration="3.777293726s" totalImagesPullingTime="1.17583902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:08.774883497 +0000 UTC m=+126.002341503" watchObservedRunningTime="2026-09-16 15:52:08.777293726 +0000 UTC m=+126.004751733"
Sep 16 15:52:09.562279 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.562245    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"]
Sep 16 15:52:09.565140 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.565123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"
Sep 16 15:52:09.567669 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.567650    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h98qv\""
Sep 16 15:52:09.573685 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.573664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"]
Sep 16 15:52:09.683344 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.683301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw5fb\" (UniqueName: \"kubernetes.io/projected/c04d4994-8249-4225-ae7f-1f60974555dd-kube-api-access-qw5fb\") pod \"network-check-source-6f8b768455-7tvxm\" (UID: \"c04d4994-8249-4225-ae7f-1f60974555dd\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"
Sep 16 15:52:09.683536 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.683371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:09.683536 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:09.683512    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 15:52:09.683536 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:09.683528    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cdf7fd65b-sdfwv: secret "image-registry-tls" not found
Sep 16 15:52:09.683687 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:09.683591    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls podName:b04d6963-a011-47b3-add8-44ac3506806f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:13.683570654 +0000 UTC m=+130.911028643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls") pod "image-registry-cdf7fd65b-sdfwv" (UID: "b04d6963-a011-47b3-add8-44ac3506806f") : secret "image-registry-tls" not found
Sep 16 15:52:09.765026 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.764996    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/0.log"
Sep 16 15:52:09.765464 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.765038    2581 generic.go:356] "Generic (PLEG): container finished" podID="c46801f9-26da-47f0-aedb-97c459b68ef8" containerID="09c6ab8fafd83a9086e89ccaa57417b050d79f6cf93440898a7284439d0ebe9c" exitCode=255
Sep 16 15:52:09.765464 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.765123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" event={"ID":"c46801f9-26da-47f0-aedb-97c459b68ef8","Type":"ContainerDied","Data":"09c6ab8fafd83a9086e89ccaa57417b050d79f6cf93440898a7284439d0ebe9c"}
Sep 16 15:52:09.765464 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.765338    2581 scope.go:117] "RemoveContainer" containerID="09c6ab8fafd83a9086e89ccaa57417b050d79f6cf93440898a7284439d0ebe9c"
Sep 16 15:52:09.766867 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.766841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw" event={"ID":"fa9bf1a4-1949-4346-bdb4-88795e371dd2","Type":"ContainerStarted","Data":"6714c61811a475cec98af670e10c0b1dea226e87c4689003d06394fa813c3f48"}
Sep 16 15:52:09.783670 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.783648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:09.783774 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.783679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw5fb\" (UniqueName: \"kubernetes.io/projected/c04d4994-8249-4225-ae7f-1f60974555dd-kube-api-access-qw5fb\") pod \"network-check-source-6f8b768455-7tvxm\" (UID: \"c04d4994-8249-4225-ae7f-1f60974555dd\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"
Sep 16 15:52:09.783830 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:09.783790    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 15:52:09.783879 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:09.783849    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls podName:107e8198-50ce-4b67-ad64-67f00fcf7e59 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:13.783832246 +0000 UTC m=+131.011290249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rvphr" (UID: "107e8198-50ce-4b67-ad64-67f00fcf7e59") : secret "samples-operator-tls" not found
Sep 16 15:52:09.792661 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.792636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw5fb\" (UniqueName: \"kubernetes.io/projected/c04d4994-8249-4225-ae7f-1f60974555dd-kube-api-access-qw5fb\") pod \"network-check-source-6f8b768455-7tvxm\" (UID: \"c04d4994-8249-4225-ae7f-1f60974555dd\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"
Sep 16 15:52:09.795669 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.795634    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw" podStartSLOduration=2.277526944 podStartE2EDuration="4.795620941s" totalImagesPullingTime="2.518093997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:09.794940438 +0000 UTC m=+127.022398444" watchObservedRunningTime="2026-09-16 15:52:09.795620941 +0000 UTC m=+127.023078968"
Sep 16 15:52:09.873592 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:09.873559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"
Sep 16 15:52:10.144714 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.144688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm"]
Sep 16 15:52:10.146762 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:10.146734    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04d4994_8249_4225_ae7f_1f60974555dd.slice/crio-99d0d97f62164b4eacae0427756beab99dced104dea28d6d2a50b8a131389913 WatchSource:0}: Error finding container 99d0d97f62164b4eacae0427756beab99dced104dea28d6d2a50b8a131389913: Status 404 returned error can't find the container with id 99d0d97f62164b4eacae0427756beab99dced104dea28d6d2a50b8a131389913
Sep 16 15:52:10.774361 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.774318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm" event={"ID":"c04d4994-8249-4225-ae7f-1f60974555dd","Type":"ContainerStarted","Data":"544434f6b1efa92fdee6af6f3ccfea1a86a8ef64d56d51b7aebfe488e47c5ac6"}
Sep 16 15:52:10.774770 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.774404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm" event={"ID":"c04d4994-8249-4225-ae7f-1f60974555dd","Type":"ContainerStarted","Data":"99d0d97f62164b4eacae0427756beab99dced104dea28d6d2a50b8a131389913"}
Sep 16 15:52:10.775858 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.775840    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/1.log"
Sep 16 15:52:10.776243 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.776228    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/0.log"
Sep 16 15:52:10.776287 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.776261    2581 generic.go:356] "Generic (PLEG): container finished" podID="c46801f9-26da-47f0-aedb-97c459b68ef8" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88" exitCode=255
Sep 16 15:52:10.776332 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.776315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" event={"ID":"c46801f9-26da-47f0-aedb-97c459b68ef8","Type":"ContainerDied","Data":"190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"}
Sep 16 15:52:10.776371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.776360    2581 scope.go:117] "RemoveContainer" containerID="09c6ab8fafd83a9086e89ccaa57417b050d79f6cf93440898a7284439d0ebe9c"
Sep 16 15:52:10.776591 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.776578    2581 scope.go:117] "RemoveContainer" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"
Sep 16 15:52:10.776797 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:10.776777    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-64ghg_openshift-console-operator(c46801f9-26da-47f0-aedb-97c459b68ef8)\"" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podUID="c46801f9-26da-47f0-aedb-97c459b68ef8"
Sep 16 15:52:10.790345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:10.790303    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-7tvxm" podStartSLOduration=1.7902878 podStartE2EDuration="1.7902878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:10.789141692 +0000 UTC m=+128.016599700" watchObservedRunningTime="2026-09-16 15:52:10.7902878 +0000 UTC m=+128.017745805"
Sep 16 15:52:11.072016 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.071929    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bwdhs"]
Sep 16 15:52:11.076221 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.076202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.078826 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.078796    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 15:52:11.078934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.078856    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 15:52:11.078934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.078864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 15:52:11.078934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.078855    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 15:52:11.078934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.078920    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ckzgw\""
Sep 16 15:52:11.086063 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.086043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bwdhs"]
Sep 16 15:52:11.194269 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.194232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f17499d-742a-4812-90b1-bed1d98b1294-crio-socket\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.194436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.194298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f17499d-742a-4812-90b1-bed1d98b1294-data-volume\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.194436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.194345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.194436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.194369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.194436 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.194414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pztw\" (UniqueName: \"kubernetes.io/projected/3f17499d-742a-4812-90b1-bed1d98b1294-kube-api-access-7pztw\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.294883 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.294825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.294883 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.294883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pztw\" (UniqueName: \"kubernetes.io/projected/3f17499d-742a-4812-90b1-bed1d98b1294-kube-api-access-7pztw\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.294906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f17499d-742a-4812-90b1-bed1d98b1294-crio-socket\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.294952    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.295031    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:11.795014678 +0000 UTC m=+129.022472661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : configmap "kube-rbac-proxy" not found
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.295037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f17499d-742a-4812-90b1-bed1d98b1294-crio-socket\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.294956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f17499d-742a-4812-90b1-bed1d98b1294-data-volume\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.295127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295443 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.295246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f17499d-742a-4812-90b1-bed1d98b1294-data-volume\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.295443 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.295248    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:11.295443 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.295313    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:11.795301457 +0000 UTC m=+129.022759441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:11.305362 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.305332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pztw\" (UniqueName: \"kubernetes.io/projected/3f17499d-742a-4812-90b1-bed1d98b1294-kube-api-access-7pztw\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.780888 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.780860    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/1.log"
Sep 16 15:52:11.781273 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.781192    2581 scope.go:117] "RemoveContainer" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"
Sep 16 15:52:11.781364 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.781345    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-64ghg_openshift-console-operator(c46801f9-26da-47f0-aedb-97c459b68ef8)\"" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podUID="c46801f9-26da-47f0-aedb-97c459b68ef8"
Sep 16 15:52:11.798685 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.798657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.798802 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:11.798699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:11.798802 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.798793    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 15:52:11.798802 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.798796    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:11.798896 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.798839    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:12.798825454 +0000 UTC m=+130.026283437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : configmap "kube-rbac-proxy" not found
Sep 16 15:52:11.798896 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:11.798850    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:12.798844813 +0000 UTC m=+130.026302797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:12.806333 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:12.806301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:12.806795 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:12.806343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:12.806795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:12.806448    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:12.806795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:12.806474    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 15:52:12.806795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:12.806515    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:14.80649889 +0000 UTC m=+132.033956876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:12.806795 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:12.806530    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:14.806523741 +0000 UTC m=+132.033981725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : configmap "kube-rbac-proxy" not found
Sep 16 15:52:13.007701 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:13.007665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:52:13.007871 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.007812    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 15:52:13.007911 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.007881    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs podName:af6df2ae-23ae-4e0f-bac7-c2b7806f29c9 nodeName:}" failed. No retries permitted until 2026-09-16 15:54:15.007865684 +0000 UTC m=+252.235323667 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs") pod "network-metrics-daemon-rv47s" (UID: "af6df2ae-23ae-4e0f-bac7-c2b7806f29c9") : secret "metrics-daemon-secret" not found
Sep 16 15:52:13.232300 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:13.232265    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rhqrq_b6a2a639-00a1-46a0-a895-bb7f6c89c275/dns-node-resolver/0.log"
Sep 16 15:52:13.718108 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:13.718061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:13.718339 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.718249    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 15:52:13.718339 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.718270    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cdf7fd65b-sdfwv: secret "image-registry-tls" not found
Sep 16 15:52:13.718449 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.718368    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls podName:b04d6963-a011-47b3-add8-44ac3506806f nodeName:}" failed. No retries permitted until 2026-09-16 15:52:21.718348454 +0000 UTC m=+138.945806453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls") pod "image-registry-cdf7fd65b-sdfwv" (UID: "b04d6963-a011-47b3-add8-44ac3506806f") : secret "image-registry-tls" not found
Sep 16 15:52:13.818419 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:13.818365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:13.818824 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.818511    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 15:52:13.818824 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:13.818587    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls podName:107e8198-50ce-4b67-ad64-67f00fcf7e59 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:21.818569828 +0000 UTC m=+139.046027811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rvphr" (UID: "107e8198-50ce-4b67-ad64-67f00fcf7e59") : secret "samples-operator-tls" not found
Sep 16 15:52:14.229673 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:14.229647    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v7nh6_1cb09db2-2a8a-48a2-8270-ac314de8cc96/node-ca/0.log"
Sep 16 15:52:14.825628 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:14.825587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:14.825628 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:14.825633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:14.826078 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:14.825735    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:14.826078 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:14.825772    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 15:52:14.826078 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:14.825802    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:18.825783592 +0000 UTC m=+136.053241576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:14.826078 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:14.825826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:18.825811286 +0000 UTC m=+136.053269271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : configmap "kube-rbac-proxy" not found
Sep 16 15:52:14.829967 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:14.829932    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hbb4s_22113a5f-d398-4e16-b11e-b96c75d5e2e9/serve-healthcheck-canary/0.log"
Sep 16 15:52:16.304013 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:16.303968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:16.304442 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:16.304027    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:16.304442 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:16.304393    2581 scope.go:117] "RemoveContainer" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"
Sep 16 15:52:16.304553 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:16.304535    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-64ghg_openshift-console-operator(c46801f9-26da-47f0-aedb-97c459b68ef8)\"" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podUID="c46801f9-26da-47f0-aedb-97c459b68ef8"
Sep 16 15:52:18.854031 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:18.853989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:18.854460 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:18.854090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:18.854460 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:18.854114    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 15:52:18.854460 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:18.854195    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:18.854460 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:18.854199    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:26.854177709 +0000 UTC m=+144.081635695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : configmap "kube-rbac-proxy" not found
Sep 16 15:52:18.854460 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:18.854258    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls podName:3f17499d-742a-4812-90b1-bed1d98b1294 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:26.854244971 +0000 UTC m=+144.081702955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwdhs" (UID: "3f17499d-742a-4812-90b1-bed1d98b1294") : secret "insights-runtime-extractor-tls" not found
Sep 16 15:52:21.777039 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:21.776991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:21.779356 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:21.779336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"image-registry-cdf7fd65b-sdfwv\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") " pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:21.878374 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:21.878329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:21.880864 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:21.880837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/107e8198-50ce-4b67-ad64-67f00fcf7e59-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rvphr\" (UID: \"107e8198-50ce-4b67-ad64-67f00fcf7e59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:21.909756 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:21.909706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:22.005598 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.005545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"
Sep 16 15:52:22.182927 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.182895    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:52:22.186326 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:22.186302    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04d6963_a011_47b3_add8_44ac3506806f.slice/crio-f427c5abdade4952f8ff7a206782bd85f5e4d61a2b72f704cd87dc5d9faa3252 WatchSource:0}: Error finding container f427c5abdade4952f8ff7a206782bd85f5e4d61a2b72f704cd87dc5d9faa3252: Status 404 returned error can't find the container with id f427c5abdade4952f8ff7a206782bd85f5e4d61a2b72f704cd87dc5d9faa3252
Sep 16 15:52:22.271212 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.271179    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr"]
Sep 16 15:52:22.812239 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.812203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr" event={"ID":"107e8198-50ce-4b67-ad64-67f00fcf7e59","Type":"ContainerStarted","Data":"29fe293c39d2be488c835fa5a84072409227431fb57b74de73e6fcdb7ae821bf"}
Sep 16 15:52:22.813596 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.813574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" event={"ID":"b04d6963-a011-47b3-add8-44ac3506806f","Type":"ContainerStarted","Data":"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"}
Sep 16 15:52:22.813679 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.813611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" event={"ID":"b04d6963-a011-47b3-add8-44ac3506806f","Type":"ContainerStarted","Data":"f427c5abdade4952f8ff7a206782bd85f5e4d61a2b72f704cd87dc5d9faa3252"}
Sep 16 15:52:22.813749 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.813728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:52:22.833935 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:22.833885    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" podStartSLOduration=17.83386885 podStartE2EDuration="17.83386885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:22.832063406 +0000 UTC m=+140.059521416" watchObservedRunningTime="2026-09-16 15:52:22.83386885 +0000 UTC m=+140.061326861"
Sep 16 15:52:24.822106 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:24.822027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr" event={"ID":"107e8198-50ce-4b67-ad64-67f00fcf7e59","Type":"ContainerStarted","Data":"407caccf9f07ca4362439daf555b570e8c60770d4b655a447b11de3ba7ce610a"}
Sep 16 15:52:24.822106 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:24.822084    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr" event={"ID":"107e8198-50ce-4b67-ad64-67f00fcf7e59","Type":"ContainerStarted","Data":"e6b4be2a66bf00e3acc26dd7b19f5260ac1987807f0a02e13a3f8cf33ea8c826"}
Sep 16 15:52:24.839460 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:24.839408    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rvphr" podStartSLOduration=16.67441588 podStartE2EDuration="18.839387404s" totalImagesPullingTime="2.164971524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:24.838474497 +0000 UTC m=+142.065932502" watchObservedRunningTime="2026-09-16 15:52:24.839387404 +0000 UTC m=+142.066845409"
Sep 16 15:52:26.921682 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:26.921617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:26.922051 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:26.921730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:26.922201 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:26.922178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f17499d-742a-4812-90b1-bed1d98b1294-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:26.923890 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:26.923874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f17499d-742a-4812-90b1-bed1d98b1294-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwdhs\" (UID: \"3f17499d-742a-4812-90b1-bed1d98b1294\") " pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:26.986097 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:26.986059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bwdhs"
Sep 16 15:52:27.254957 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.254879    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bwdhs"]
Sep 16 15:52:27.257743 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:27.257707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f17499d_742a_4812_90b1_bed1d98b1294.slice/crio-2a83aa859fd3870eb5e210865ea267f586fdedb89eefa41c55c3dad5a9620e06 WatchSource:0}: Error finding container 2a83aa859fd3870eb5e210865ea267f586fdedb89eefa41c55c3dad5a9620e06: Status 404 returned error can't find the container with id 2a83aa859fd3870eb5e210865ea267f586fdedb89eefa41c55c3dad5a9620e06
Sep 16 15:52:27.304173 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.304140    2581 scope.go:117] "RemoveContainer" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"
Sep 16 15:52:27.832994 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.832942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwdhs" event={"ID":"3f17499d-742a-4812-90b1-bed1d98b1294","Type":"ContainerStarted","Data":"cb171cbf638fcb7adbb42945963ba58303f4540cdf558a53201bbf4eaccf9cc9"}
Sep 16 15:52:27.833145 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.833015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwdhs" event={"ID":"3f17499d-742a-4812-90b1-bed1d98b1294","Type":"ContainerStarted","Data":"2a83aa859fd3870eb5e210865ea267f586fdedb89eefa41c55c3dad5a9620e06"}
Sep 16 15:52:27.834699 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.834681    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 15:52:27.835146 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.835123    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/1.log"
Sep 16 15:52:27.835244 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.835185    2581 generic.go:356] "Generic (PLEG): container finished" podID="c46801f9-26da-47f0-aedb-97c459b68ef8" containerID="8ac905faea01c9f34da64c405af37d1b33fbc72c26664262aee6a5e2b2fe7306" exitCode=255
Sep 16 15:52:27.835244 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.835231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" event={"ID":"c46801f9-26da-47f0-aedb-97c459b68ef8","Type":"ContainerDied","Data":"8ac905faea01c9f34da64c405af37d1b33fbc72c26664262aee6a5e2b2fe7306"}
Sep 16 15:52:27.835343 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.835266    2581 scope.go:117] "RemoveContainer" containerID="190c84342b4b6132d7eebbf6b8bd473adf74a33ec4cb4ed2ee7b8c116efccd88"
Sep 16 15:52:27.835734 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:27.835715    2581 scope.go:117] "RemoveContainer" containerID="8ac905faea01c9f34da64c405af37d1b33fbc72c26664262aee6a5e2b2fe7306"
Sep 16 15:52:27.836063 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:27.835935    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-64ghg_openshift-console-operator(c46801f9-26da-47f0-aedb-97c459b68ef8)\"" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podUID="c46801f9-26da-47f0-aedb-97c459b68ef8"
Sep 16 15:52:28.840892 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:28.840848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwdhs" event={"ID":"3f17499d-742a-4812-90b1-bed1d98b1294","Type":"ContainerStarted","Data":"8037d13cfcc595ea828a4f7086c22373f927b114897cc3e5dbf04b62f3920618"}
Sep 16 15:52:28.842578 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:28.842550    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 15:52:29.847863 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:29.847826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwdhs" event={"ID":"3f17499d-742a-4812-90b1-bed1d98b1294","Type":"ContainerStarted","Data":"5605100ce9b30c160e6faf4072ecf9a1afe3ce8802f2d3fd400f1f0d97c56557"}
Sep 16 15:52:29.867438 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:29.867389    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bwdhs" podStartSLOduration=16.898018344 podStartE2EDuration="18.867372332s" totalImagesPullingTime="1.969353988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 15:52:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:29.865704424 +0000 UTC m=+147.093162429" watchObservedRunningTime="2026-09-16 15:52:29.867372332 +0000 UTC m=+147.094830338"
Sep 16 15:52:36.303633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.303599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:36.303998 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.303653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:36.303998 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.303944    2581 scope.go:117] "RemoveContainer" containerID="8ac905faea01c9f34da64c405af37d1b33fbc72c26664262aee6a5e2b2fe7306"
Sep 16 15:52:36.304125 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:36.304102    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-64ghg_openshift-console-operator(c46801f9-26da-47f0-aedb-97c459b68ef8)\"" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podUID="c46801f9-26da-47f0-aedb-97c459b68ef8"
Sep 16 15:52:36.316765 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.316737    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:52:36.329259 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.329231    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"]
Sep 16 15:52:36.332361 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.332346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.336050 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.336027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 15:52:36.336204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.336050    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 15:52:36.336204 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.336031    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sd7jv\""
Sep 16 15:52:36.345412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.345387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"]
Sep 16 15:52:36.394651 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.394434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.396096 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.394866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.436589 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.436560    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dfd56794f-w8frz"]
Sep 16 15:52:36.439342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.439327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.452633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.452608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dfd56794f-w8frz"]
Sep 16 15:52:36.495358 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.495320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.495536 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.495365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.496010 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.495988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.497761 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.497738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72cacd54-d295-4c94-b9bd-79cd8c9fd6b9-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-pbrk9\" (UID: \"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.596108 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-tls\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596108 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b60d0094-bb1b-43a3-a346-fc66a95268ba-ca-trust-extracted\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596108 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-certificates\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596108 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-installation-pull-secrets\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-bound-sa-token\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-trusted-ca\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn8hf\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-kube-api-access-gn8hf\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.596371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.596292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-image-registry-private-configuration\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.640913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.640885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"
Sep 16 15:52:36.697740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn8hf\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-kube-api-access-gn8hf\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-image-registry-private-configuration\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-tls\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b60d0094-bb1b-43a3-a346-fc66a95268ba-ca-trust-extracted\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-certificates\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-installation-pull-secrets\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-bound-sa-token\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.697934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.697912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-trusted-ca\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.699338 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.698323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b60d0094-bb1b-43a3-a346-fc66a95268ba-ca-trust-extracted\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.699338 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.699064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-certificates\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.699338 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.699294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b60d0094-bb1b-43a3-a346-fc66a95268ba-trusted-ca\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.702049 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.701664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-image-registry-private-configuration\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.702049 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.702008    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-registry-tls\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.703591 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.703288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b60d0094-bb1b-43a3-a346-fc66a95268ba-installation-pull-secrets\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.711185 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.711132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn8hf\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-kube-api-access-gn8hf\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.711473 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.711456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b60d0094-bb1b-43a3-a346-fc66a95268ba-bound-sa-token\") pod \"image-registry-7dfd56794f-w8frz\" (UID: \"b60d0094-bb1b-43a3-a346-fc66a95268ba\") " pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.747735 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.747714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:36.909924 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:36.909893    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9"]
Sep 16 15:52:36.913258 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:36.913231    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72cacd54_d295_4c94_b9bd_79cd8c9fd6b9.slice/crio-1a884d1c4c4f2a6fc96cffdb4b336ec86bd13f5b61eef8a6684e449c88bbd836 WatchSource:0}: Error finding container 1a884d1c4c4f2a6fc96cffdb4b336ec86bd13f5b61eef8a6684e449c88bbd836: Status 404 returned error can't find the container with id 1a884d1c4c4f2a6fc96cffdb4b336ec86bd13f5b61eef8a6684e449c88bbd836
Sep 16 15:52:37.036111 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.036022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dfd56794f-w8frz"]
Sep 16 15:52:37.038607 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:37.038582    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60d0094_bb1b_43a3_a346_fc66a95268ba.slice/crio-b933bb48e85f92d9232a6de920ccb73fb87bcf1355065b2d154714e200d257ba WatchSource:0}: Error finding container b933bb48e85f92d9232a6de920ccb73fb87bcf1355065b2d154714e200d257ba: Status 404 returned error can't find the container with id b933bb48e85f92d9232a6de920ccb73fb87bcf1355065b2d154714e200d257ba
Sep 16 15:52:37.875205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.875148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz" event={"ID":"b60d0094-bb1b-43a3-a346-fc66a95268ba","Type":"ContainerStarted","Data":"9f94a5d55efaebeb20c4a99dd868a8e65982e4876a402dbc50342618f0c31005"}
Sep 16 15:52:37.875660 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.875229    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:37.875660 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.875247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz" event={"ID":"b60d0094-bb1b-43a3-a346-fc66a95268ba","Type":"ContainerStarted","Data":"b933bb48e85f92d9232a6de920ccb73fb87bcf1355065b2d154714e200d257ba"}
Sep 16 15:52:37.876387 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.876358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9" event={"ID":"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9","Type":"ContainerStarted","Data":"1a884d1c4c4f2a6fc96cffdb4b336ec86bd13f5b61eef8a6684e449c88bbd836"}
Sep 16 15:52:37.896457 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:37.896413    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz" podStartSLOduration=1.8963971800000001 podStartE2EDuration="1.89639718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:37.894550179 +0000 UTC m=+155.122008199" watchObservedRunningTime="2026-09-16 15:52:37.89639718 +0000 UTC m=+155.123855185"
Sep 16 15:52:38.881022 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:38.880981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9" event={"ID":"72cacd54-d295-4c94-b9bd-79cd8c9fd6b9","Type":"ContainerStarted","Data":"b5d1fe8f164feb5c84ede2dd87b39e61c612979b80e166b7f6bf2a5b51c51336"}
Sep 16 15:52:38.898494 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:38.898446    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-pbrk9" podStartSLOduration=1.645172992 podStartE2EDuration="2.898430452s" totalImagesPullingTime="1.25325746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:38.897141355 +0000 UTC m=+156.124599377" watchObservedRunningTime="2026-09-16 15:52:38.898430452 +0000 UTC m=+156.125888458"
Sep 16 15:52:40.450491 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.450451    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"]
Sep 16 15:52:40.453886 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.453870    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:40.457037 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.457014    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-578st\""
Sep 16 15:52:40.457195 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.457052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 16 15:52:40.463148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.463125    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"]
Sep 16 15:52:40.526611 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.526565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-fh77c\" (UID: \"bf521d31-663b-41a2-8626-6bc53748c479\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:40.627008 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.626966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-fh77c\" (UID: \"bf521d31-663b-41a2-8626-6bc53748c479\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:40.627212 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:40.627114    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 15:52:40.627265 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:40.627223    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates podName:bf521d31-663b-41a2-8626-6bc53748c479 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:41.12720002 +0000 UTC m=+158.354658004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-fh77c" (UID: "bf521d31-663b-41a2-8626-6bc53748c479") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 15:52:40.630959 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:40.630921    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vdwrr" podUID="39025b22-64ea-4d4b-a568-69ccf39c1e3f"
Sep 16 15:52:40.887107 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:40.887078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:52:41.131472 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:41.131443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-fh77c\" (UID: \"bf521d31-663b-41a2-8626-6bc53748c479\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:41.133837 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:41.133819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bf521d31-663b-41a2-8626-6bc53748c479-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-fh77c\" (UID: \"bf521d31-663b-41a2-8626-6bc53748c479\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:41.362466 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:41.362420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:41.636171 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:41.636126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"]
Sep 16 15:52:41.639551 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:41.639524    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf521d31_663b_41a2_8626_6bc53748c479.slice/crio-11401ac27be943ba3b8a28cefa62e9964327ca34a7937c193ef092d5b47947dd WatchSource:0}: Error finding container 11401ac27be943ba3b8a28cefa62e9964327ca34a7937c193ef092d5b47947dd: Status 404 returned error can't find the container with id 11401ac27be943ba3b8a28cefa62e9964327ca34a7937c193ef092d5b47947dd
Sep 16 15:52:41.891518 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:41.891484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c" event={"ID":"bf521d31-663b-41a2-8626-6bc53748c479","Type":"ContainerStarted","Data":"11401ac27be943ba3b8a28cefa62e9964327ca34a7937c193ef092d5b47947dd"}
Sep 16 15:52:42.320038 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:42.319952    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rv47s" podUID="af6df2ae-23ae-4e0f-bac7-c2b7806f29c9"
Sep 16 15:52:43.899603 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:43.899565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c" event={"ID":"bf521d31-663b-41a2-8626-6bc53748c479","Type":"ContainerStarted","Data":"21450ca20b3165fde64176e21070d8e1c4d9f0fa1e7ab39aa9283fa42cfa1164"}
Sep 16 15:52:43.900062 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:43.899626    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:43.904181 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:43.904141    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c"
Sep 16 15:52:43.915969 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:43.915928    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-fh77c" podStartSLOduration=2.5117922 podStartE2EDuration="3.915916421s" totalImagesPullingTime="1.404124221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:43.915607196 +0000 UTC m=+161.143065202" watchObservedRunningTime="2026-09-16 15:52:43.915916421 +0000 UTC m=+161.143374426"
Sep 16 15:52:44.563503 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.563470    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"]
Sep 16 15:52:44.566211 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.566195    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.570080 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570056    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 15:52:44.570272 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570084    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 15:52:44.570272 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570146    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mq8bt\""
Sep 16 15:52:44.570272 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 15:52:44.570272 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570192    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 15:52:44.570468 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.570421    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 15:52:44.575987 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.575965    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"]
Sep 16 15:52:44.662500 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.662459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.662708 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.662530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.662708 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.662561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bdafc9f-0860-46eb-959e-251343192dc1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.662708 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.662632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptzn5\" (UniqueName: \"kubernetes.io/projected/8bdafc9f-0860-46eb-959e-251343192dc1-kube-api-access-ptzn5\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.763460 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.763426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.763624 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.763487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.763624 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.763504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bdafc9f-0860-46eb-959e-251343192dc1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.763624 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.763527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptzn5\" (UniqueName: \"kubernetes.io/projected/8bdafc9f-0860-46eb-959e-251343192dc1-kube-api-access-ptzn5\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.764271 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.764245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bdafc9f-0860-46eb-959e-251343192dc1-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.765952 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.765927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.766051 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.765994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8bdafc9f-0860-46eb-959e-251343192dc1-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.772685 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.772665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptzn5\" (UniqueName: \"kubernetes.io/projected/8bdafc9f-0860-46eb-959e-251343192dc1-kube-api-access-ptzn5\") pod \"prometheus-operator-68bcd7f987-s99hx\" (UID: \"8bdafc9f-0860-46eb-959e-251343192dc1\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:44.874921 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:44.874846    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"
Sep 16 15:52:45.146571 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.146532    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s99hx"]
Sep 16 15:52:45.150590 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:45.150565    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdafc9f_0860_46eb_959e_251343192dc1.slice/crio-94e2d5f67a8d3d99c5f8866e6023f22579a6bec94125055aa0e81e4f5b895254 WatchSource:0}: Error finding container 94e2d5f67a8d3d99c5f8866e6023f22579a6bec94125055aa0e81e4f5b895254: Status 404 returned error can't find the container with id 94e2d5f67a8d3d99c5f8866e6023f22579a6bec94125055aa0e81e4f5b895254
Sep 16 15:52:45.571905 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.571816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:52:45.574091 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.574070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39025b22-64ea-4d4b-a568-69ccf39c1e3f-metrics-tls\") pod \"dns-default-vdwrr\" (UID: \"39025b22-64ea-4d4b-a568-69ccf39c1e3f\") " pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:52:45.691224 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.691190    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-thq8g\""
Sep 16 15:52:45.699282 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.699262    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:52:45.908263 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.908187    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx" event={"ID":"8bdafc9f-0860-46eb-959e-251343192dc1","Type":"ContainerStarted","Data":"94e2d5f67a8d3d99c5f8866e6023f22579a6bec94125055aa0e81e4f5b895254"}
Sep 16 15:52:45.970905 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:45.970670    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdwrr"]
Sep 16 15:52:45.973602 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:45.973569    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39025b22_64ea_4d4b_a568_69ccf39c1e3f.slice/crio-11bd67ba17c146d2994ee565bac928674319f01062a1908891d5d7a418546f43 WatchSource:0}: Error finding container 11bd67ba17c146d2994ee565bac928674319f01062a1908891d5d7a418546f43: Status 404 returned error can't find the container with id 11bd67ba17c146d2994ee565bac928674319f01062a1908891d5d7a418546f43
Sep 16 15:52:46.323014 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.322919    2581 patch_prober.go:28] interesting pod/image-registry-cdf7fd65b-sdfwv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 15:52:46.323014 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.322993    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 15:52:46.914411 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.914368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx" event={"ID":"8bdafc9f-0860-46eb-959e-251343192dc1","Type":"ContainerStarted","Data":"9cc15e6a3d760290bb06172b1ff134c4fe731cf5c9e13246b7b47890b9cfca95"}
Sep 16 15:52:46.914598 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.914467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx" event={"ID":"8bdafc9f-0860-46eb-959e-251343192dc1","Type":"ContainerStarted","Data":"9999a44a6f026750eaba3cf354e93e2444581dc474fbb053be2f4b76e7c39f08"}
Sep 16 15:52:46.915809 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.915773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdwrr" event={"ID":"39025b22-64ea-4d4b-a568-69ccf39c1e3f","Type":"ContainerStarted","Data":"11bd67ba17c146d2994ee565bac928674319f01062a1908891d5d7a418546f43"}
Sep 16 15:52:46.931280 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:46.931227    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s99hx" podStartSLOduration=1.518889922 podStartE2EDuration="2.931208734s" totalImagesPullingTime="1.412318812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:46.931005113 +0000 UTC m=+164.158463119" watchObservedRunningTime="2026-09-16 15:52:46.931208734 +0000 UTC m=+164.158666743"
Sep 16 15:52:47.921475 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:47.921434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdwrr" event={"ID":"39025b22-64ea-4d4b-a568-69ccf39c1e3f","Type":"ContainerStarted","Data":"453663182caaeea8fe6cd4bbc209b5fad1879af4ec575f8b74d922975bc367fe"}
Sep 16 15:52:47.921934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:47.921500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdwrr" event={"ID":"39025b22-64ea-4d4b-a568-69ccf39c1e3f","Type":"ContainerStarted","Data":"6a41f06bf0757afa743bf108f40eed5e80de909807add1eb2e8cdc626455f62a"}
Sep 16 15:52:47.940835 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:47.940782    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vdwrr" podStartSLOduration=129.218088128 podStartE2EDuration="2m10.940765441s" totalImagesPullingTime="1.722677313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:47.939562802 +0000 UTC m=+165.167020807" watchObservedRunningTime="2026-09-16 15:52:47.940765441 +0000 UTC m=+165.168223482"
Sep 16 15:52:48.924694 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.924663    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:52:48.973305 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.973270    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"]
Sep 16 15:52:48.976629 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.976608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:48.979871 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.979849    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 15:52:48.979986 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.979968    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 15:52:48.981232 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.981217    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-h5z8x\""
Sep 16 15:52:48.981303 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:48.981221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 15:52:49.011680 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.011626    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"]
Sep 16 15:52:49.012637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.012612    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5vzf9"]
Sep 16 15:52:49.015789 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.015768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.018825 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.018806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 15:52:49.019119 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.019091    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 15:52:49.019240 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.019148    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fzfxs\""
Sep 16 15:52:49.019240 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.019222    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 15:52:49.103464 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-wtmp\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkpjf\" (UniqueName: \"kubernetes.io/projected/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-api-access-gkpjf\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.103633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-tls\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-metrics-client-ca\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm48t\" (UniqueName: \"kubernetes.io/projected/7b7cc6e7-d253-46c8-b6be-6de16054d773-kube-api-access-dm48t\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103840 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.103840 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103840 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-root\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103840 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-textfile\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103983 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-sys\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.103983 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.103983 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8142c874-9e64-4134-8c29-68e5156c1e1a-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.103983 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.103983 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.104153 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.103988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205192 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205192 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-wtmp\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205192 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpjf\" (UniqueName: \"kubernetes.io/projected/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-api-access-gkpjf\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-tls\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-metrics-client-ca\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm48t\" (UniqueName: \"kubernetes.io/projected/7b7cc6e7-d253-46c8-b6be-6de16054d773-kube-api-access-dm48t\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-wtmp\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-root\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205435 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:49.205388    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-textfile\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:49.205458    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls podName:8142c874-9e64-4134-8c29-68e5156c1e1a nodeName:}" failed. No retries permitted until 2026-09-16 15:52:49.705439797 +0000 UTC m=+166.932897800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-nqbj5" (UID: "8142c874-9e64-4134-8c29-68e5156c1e1a") : secret "kube-state-metrics-tls" not found
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-sys\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-root\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8142c874-9e64-4134-8c29-68e5156c1e1a-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7cc6e7-d253-46c8-b6be-6de16054d773-sys\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.205819 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.206251 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.205939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-metrics-client-ca\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.206251 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.206004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8142c874-9e64-4134-8c29-68e5156c1e1a-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.206350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.206246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-accelerators-collector-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.206350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.206254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-textfile\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.206350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.206262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.206350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.206268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.208004 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.207978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.208124 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.208059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.208242 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.208219    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7cc6e7-d253-46c8-b6be-6de16054d773-node-exporter-tls\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.214042 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.214023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm48t\" (UniqueName: \"kubernetes.io/projected/7b7cc6e7-d253-46c8-b6be-6de16054d773-kube-api-access-dm48t\") pod \"node-exporter-5vzf9\" (UID: \"7b7cc6e7-d253-46c8-b6be-6de16054d773\") " pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.214723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.214700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpjf\" (UniqueName: \"kubernetes.io/projected/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-api-access-gkpjf\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.325533 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.325479    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5vzf9"
Sep 16 15:52:49.337503 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:49.337476    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7cc6e7_d253_46c8_b6be_6de16054d773.slice/crio-2d89a8b8c8dc408bd003c30152f7c6fa9b0eac7a07b0ed8434eceddedad459ce WatchSource:0}: Error finding container 2d89a8b8c8dc408bd003c30152f7c6fa9b0eac7a07b0ed8434eceddedad459ce: Status 404 returned error can't find the container with id 2d89a8b8c8dc408bd003c30152f7c6fa9b0eac7a07b0ed8434eceddedad459ce
Sep 16 15:52:49.709231 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.709197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.711676 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.711641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8142c874-9e64-4134-8c29-68e5156c1e1a-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-nqbj5\" (UID: \"8142c874-9e64-4134-8c29-68e5156c1e1a\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.886258 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.886212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"
Sep 16 15:52:49.932431 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:49.932343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vzf9" event={"ID":"7b7cc6e7-d253-46c8-b6be-6de16054d773","Type":"ContainerStarted","Data":"2d89a8b8c8dc408bd003c30152f7c6fa9b0eac7a07b0ed8434eceddedad459ce"}
Sep 16 15:52:50.170213 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.170175    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5"]
Sep 16 15:52:50.284706 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:50.284675    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8142c874_9e64_4134_8c29_68e5156c1e1a.slice/crio-ce51d3ef33847496ddaf482356f420fff37fb740f1d302d96eecd0184588b8ec WatchSource:0}: Error finding container ce51d3ef33847496ddaf482356f420fff37fb740f1d302d96eecd0184588b8ec: Status 404 returned error can't find the container with id ce51d3ef33847496ddaf482356f420fff37fb740f1d302d96eecd0184588b8ec
Sep 16 15:52:50.304632 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.304608    2581 scope.go:117] "RemoveContainer" containerID="8ac905faea01c9f34da64c405af37d1b33fbc72c26664262aee6a5e2b2fe7306"
Sep 16 15:52:50.937574 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.937547    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 15:52:50.938034 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.937651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" event={"ID":"c46801f9-26da-47f0-aedb-97c459b68ef8","Type":"ContainerStarted","Data":"b397663b445bd125d1c4c5acbe33b25d396ba77505156506bf3dc5b528db2374"}
Sep 16 15:52:50.938034 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.937933    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:50.939524 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.939500    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b7cc6e7-d253-46c8-b6be-6de16054d773" containerID="698f8189150027c44a05260f4afc25ef13ee527f1e24c7a05afc25f35743f71a" exitCode=0
Sep 16 15:52:50.939649 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.939565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vzf9" event={"ID":"7b7cc6e7-d253-46c8-b6be-6de16054d773","Type":"ContainerDied","Data":"698f8189150027c44a05260f4afc25ef13ee527f1e24c7a05afc25f35743f71a"}
Sep 16 15:52:50.941062 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.941034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5" event={"ID":"8142c874-9e64-4134-8c29-68e5156c1e1a","Type":"ContainerStarted","Data":"ce51d3ef33847496ddaf482356f420fff37fb740f1d302d96eecd0184588b8ec"}
Sep 16 15:52:50.943018 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.942994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-64ghg"
Sep 16 15:52:50.955967 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:50.955924    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-64ghg" podStartSLOduration=43.5514222 podStartE2EDuration="45.955909433s" totalImagesPullingTime="2.404487233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:50.954675506 +0000 UTC m=+168.182133512" watchObservedRunningTime="2026-09-16 15:52:50.955909433 +0000 UTC m=+168.183367490"
Sep 16 15:52:51.952350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:51.952296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vzf9" event={"ID":"7b7cc6e7-d253-46c8-b6be-6de16054d773","Type":"ContainerStarted","Data":"108efa6f688e8599a216b12ba691702b49cd332bf63e90d08eb9a48c4590fcc3"}
Sep 16 15:52:51.952803 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:51.952400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5vzf9" event={"ID":"7b7cc6e7-d253-46c8-b6be-6de16054d773","Type":"ContainerStarted","Data":"de020efb6b391e11a9c401a28308a63d1b7c1ccf204364f45523fe59e972d4e5"}
Sep 16 15:52:51.972213 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:51.972152    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5vzf9" podStartSLOduration=2.91676934 podStartE2EDuration="3.97213699s" totalImagesPullingTime="992.029685ms" totalInitContainerRuntime="63.337965ms" isStatefulPod=true podCreationTimestamp="2026-09-16 15:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:51.970083569 +0000 UTC m=+169.197541590" watchObservedRunningTime="2026-09-16 15:52:51.97213699 +0000 UTC m=+169.199594995"
Sep 16 15:52:52.958093 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:52.958056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5" event={"ID":"8142c874-9e64-4134-8c29-68e5156c1e1a","Type":"ContainerStarted","Data":"44585627d4efe05b1c5ca5c09131194acf8831b241fd11130cc661c7cb2c7c33"}
Sep 16 15:52:52.958549 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:52.958119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5" event={"ID":"8142c874-9e64-4134-8c29-68e5156c1e1a","Type":"ContainerStarted","Data":"ad60fd8f677b0d1379a2077ce515036db92746229de9d85a9921344c0d5af64b"}
Sep 16 15:52:52.958549 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:52.958139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5" event={"ID":"8142c874-9e64-4134-8c29-68e5156c1e1a","Type":"ContainerStarted","Data":"a66be408f77889100078d642d3a36a5ae9f3d6a31579629b630fceef65e3dfd0"}
Sep 16 15:52:52.982530 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:52.982478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-nqbj5" podStartSLOduration=3.011466538 podStartE2EDuration="4.982462276s" totalImagesPullingTime="1.970995738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:52.980925603 +0000 UTC m=+170.208383610" watchObservedRunningTime="2026-09-16 15:52:52.982462276 +0000 UTC m=+170.209920282"
Sep 16 15:52:53.689551 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.689514    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"]
Sep 16 15:52:53.692716 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.692695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:53.695375 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.695348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jwlp7\""
Sep 16 15:52:53.695488 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.695341    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 16 15:52:53.701001 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.700978    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"]
Sep 16 15:52:53.845227 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.845195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndn9\" (UniqueName: \"kubernetes.io/projected/e865b252-175e-4808-8fa9-c81e392f6ce5-kube-api-access-lndn9\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:53.845397 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.845290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:53.946586 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.946493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:53.946586 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.946549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndn9\" (UniqueName: \"kubernetes.io/projected/e865b252-175e-4808-8fa9-c81e392f6ce5-kube-api-access-lndn9\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:53.946803 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:53.946649    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 16 15:52:53.946803 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:52:53.946719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert podName:e865b252-175e-4808-8fa9-c81e392f6ce5 nodeName:}" failed. No retries permitted until 2026-09-16 15:52:54.446701806 +0000 UTC m=+171.674159798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-578pw" (UID: "e865b252-175e-4808-8fa9-c81e392f6ce5") : secret "monitoring-plugin-cert" not found
Sep 16 15:52:53.955659 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:53.955626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndn9\" (UniqueName: \"kubernetes.io/projected/e865b252-175e-4808-8fa9-c81e392f6ce5-kube-api-access-lndn9\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:54.450330 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:54.450294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:54.452727 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:54.452706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e865b252-175e-4808-8fa9-c81e392f6ce5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-578pw\" (UID: \"e865b252-175e-4808-8fa9-c81e392f6ce5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:54.602292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:54.602255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:54.867562 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:54.867411    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"]
Sep 16 15:52:54.870279 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:52:54.870246    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode865b252_175e_4808_8fa9_c81e392f6ce5.slice/crio-2e9e40f61047b2b79e9a8850d35ad58840663de56fc9efb49f70c3d476a2553f WatchSource:0}: Error finding container 2e9e40f61047b2b79e9a8850d35ad58840663de56fc9efb49f70c3d476a2553f: Status 404 returned error can't find the container with id 2e9e40f61047b2b79e9a8850d35ad58840663de56fc9efb49f70c3d476a2553f
Sep 16 15:52:54.971453 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:54.971411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw" event={"ID":"e865b252-175e-4808-8fa9-c81e392f6ce5","Type":"ContainerStarted","Data":"2e9e40f61047b2b79e9a8850d35ad58840663de56fc9efb49f70c3d476a2553f"}
Sep 16 15:52:55.304726 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:55.304688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:52:56.321371 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.321344    2581 patch_prober.go:28] interesting pod/image-registry-cdf7fd65b-sdfwv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 15:52:56.321649 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.321386    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 15:52:56.752934 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.752896    2581 patch_prober.go:28] interesting pod/image-registry-7dfd56794f-w8frz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 15:52:56.753104 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.752970    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz" podUID="b60d0094-bb1b-43a3-a346-fc66a95268ba" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 15:52:56.982129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.982087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw" event={"ID":"e865b252-175e-4808-8fa9-c81e392f6ce5","Type":"ContainerStarted","Data":"1b6bcc25faa49b62ae914bc1254cc53f9271379667c15bdf6990a973cbb94659"}
Sep 16 15:52:56.982313 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.982247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:56.986814 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.986791    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw"
Sep 16 15:52:56.997000 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:56.996950    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-578pw" podStartSLOduration=2.609043894 podStartE2EDuration="3.996933997s" totalImagesPullingTime="1.387890103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:52:56.996564222 +0000 UTC m=+174.224022242" watchObservedRunningTime="2026-09-16 15:52:56.996933997 +0000 UTC m=+174.224392004"
Sep 16 15:52:58.887080 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:58.887050    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dfd56794f-w8frz"
Sep 16 15:52:58.934922 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:52:58.934894    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vdwrr"
Sep 16 15:53:01.335837 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.335772    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry" containerID="cri-o://02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84" gracePeriod=30
Sep 16 15:53:01.574859 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.574835    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:53:01.591951 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.591864    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:01.592193 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.592179    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry"
Sep 16 15:53:01.592241 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.592194    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry"
Sep 16 15:53:01.592275 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.592257    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04d6963-a011-47b3-add8-44ac3506806f" containerName="registry"
Sep 16 15:53:01.595044 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.595023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.598199 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 15:53:01.598199 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 15:53:01.598378 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598250    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 15:53:01.598378 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598181    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 15:53:01.598378 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598328    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 15:53:01.598498 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.598381    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wzgsr\""
Sep 16 15:53:01.599532 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.599515    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 15:53:01.599532 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.599532    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 15:53:01.606598 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.606574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:01.607254 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607232    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607345 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607322    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607448 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607448 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607386    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607448 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607425    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607597 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607480    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607597 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607516    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkpzm\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm\") pod \"b04d6963-a011-47b3-add8-44ac3506806f\" (UID: \"b04d6963-a011-47b3-add8-44ac3506806f\") "
Sep 16 15:53:01.607703 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.607760 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.607816 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqs55\" (UniqueName: \"kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.607816 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.607909 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.607909 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.607858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.608262 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.608181    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:53:01.608970 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.608947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:53:01.611195 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.611151    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:53:01.611346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.611314    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:53:01.611428 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.611365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm" (OuterVolumeSpecName: "kube-api-access-kkpzm") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "kube-api-access-kkpzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:53:01.611428 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.611381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:53:01.611726 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.611703    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:53:01.617689 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.617665    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b04d6963-a011-47b3-add8-44ac3506806f" (UID: "b04d6963-a011-47b3-add8-44ac3506806f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 15:53:01.709066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqs55\" (UniqueName: \"kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.709278 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709271    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b04d6963-a011-47b3-add8-44ac3506806f-ca-trust-extracted\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709287    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkpzm\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-kube-api-access-kkpzm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709301    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-bound-sa-token\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709313    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-registry-certificates\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709325    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b04d6963-a011-47b3-add8-44ac3506806f-trusted-ca\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709338    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-image-registry-private-configuration\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709350    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b04d6963-a011-47b3-add8-44ac3506806f-installation-pull-secrets\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.709595 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709362    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b04d6963-a011-47b3-add8-44ac3506806f-registry-tls\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:01.710023 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.709997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.710092 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.710069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.710126 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.710069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.711547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.711525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.711639 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.711560    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.718387 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.718367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqs55\" (UniqueName: \"kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55\") pod \"console-5c5f4f5db4-5rnql\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") " pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:01.903997 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:01.903962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:02.003577 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.003544    2581 generic.go:356] "Generic (PLEG): container finished" podID="b04d6963-a011-47b3-add8-44ac3506806f" containerID="02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84" exitCode=0
Sep 16 15:53:02.003737 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.003590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv"
Sep 16 15:53:02.003737 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.003632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" event={"ID":"b04d6963-a011-47b3-add8-44ac3506806f","Type":"ContainerDied","Data":"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"}
Sep 16 15:53:02.003737 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.003685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cdf7fd65b-sdfwv" event={"ID":"b04d6963-a011-47b3-add8-44ac3506806f","Type":"ContainerDied","Data":"f427c5abdade4952f8ff7a206782bd85f5e4d61a2b72f704cd87dc5d9faa3252"}
Sep 16 15:53:02.003737 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.003706    2581 scope.go:117] "RemoveContainer" containerID="02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"
Sep 16 15:53:02.011845 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.011828    2581 scope.go:117] "RemoveContainer" containerID="02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"
Sep 16 15:53:02.012130 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:53:02.012111    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84\": container with ID starting with 02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84 not found: ID does not exist" containerID="02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"
Sep 16 15:53:02.012221 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.012144    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84"} err="failed to get container status \"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84\": rpc error: code = NotFound desc = could not find container \"02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84\": container with ID starting with 02460f2d88a5a0b9e0a672edd12df4ded375e450b510f09bbeb684f1cc60ae84 not found: ID does not exist"
Sep 16 15:53:02.023782 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.023756    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:53:02.028216 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.028195    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-cdf7fd65b-sdfwv"]
Sep 16 15:53:02.205033 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:02.205006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:02.207440 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:53:02.207417    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1691c616_7a3d_4ca7_92aa_55403856f303.slice/crio-2dcf1f1ba604c2ef733025f9f9fd5d02b4048841b60546721e9d8c584d843ef6 WatchSource:0}: Error finding container 2dcf1f1ba604c2ef733025f9f9fd5d02b4048841b60546721e9d8c584d843ef6: Status 404 returned error can't find the container with id 2dcf1f1ba604c2ef733025f9f9fd5d02b4048841b60546721e9d8c584d843ef6
Sep 16 15:53:03.010652 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:03.010618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f4f5db4-5rnql" event={"ID":"1691c616-7a3d-4ca7-92aa-55403856f303","Type":"ContainerStarted","Data":"2dcf1f1ba604c2ef733025f9f9fd5d02b4048841b60546721e9d8c584d843ef6"}
Sep 16 15:53:03.313670 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:03.313587    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04d6963-a011-47b3-add8-44ac3506806f" path="/var/lib/kubelet/pods/b04d6963-a011-47b3-add8-44ac3506806f/volumes"
Sep 16 15:53:06.022493 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:06.022449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f4f5db4-5rnql" event={"ID":"1691c616-7a3d-4ca7-92aa-55403856f303","Type":"ContainerStarted","Data":"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"}
Sep 16 15:53:06.039589 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:06.039545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5f4f5db4-5rnql" podStartSLOduration=2.163348601 podStartE2EDuration="5.039530542s" totalImagesPullingTime="2.876181941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:53:06.039114741 +0000 UTC m=+183.266572747" watchObservedRunningTime="2026-09-16 15:53:06.039530542 +0000 UTC m=+183.266988547"
Sep 16 15:53:10.413176 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.413123    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:53:10.420652 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.420628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.429316 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.429290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:53:10.431370 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.431343    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 15:53:10.485331 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485331 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485536 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485403    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485536 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl66d\" (UniqueName: \"kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485623 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485623 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.485700 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.485633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586555 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl66d\" (UniqueName: \"kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586733 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586733 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586733 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586883 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586883 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.586883 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.586870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.587292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.587275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.587740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.587712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.588067 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.588040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.588150 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.588067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.589206 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.589184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.589576 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.589558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.594788 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.594765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl66d\" (UniqueName: \"kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d\") pod \"console-6fc644567f-vpl46\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") " pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:10.730099 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:10.730022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:11.008790 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:11.008767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:53:11.011646 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:53:11.011618    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5faa63bb_6207_4048_a6e3_d19d611aef45.slice/crio-9f078fcbe6e026d843f472b1be14c2cd021b643a71195fbdf5cefb101de1ad0e WatchSource:0}: Error finding container 9f078fcbe6e026d843f472b1be14c2cd021b643a71195fbdf5cefb101de1ad0e: Status 404 returned error can't find the container with id 9f078fcbe6e026d843f472b1be14c2cd021b643a71195fbdf5cefb101de1ad0e
Sep 16 15:53:11.041526 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:11.041498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc644567f-vpl46" event={"ID":"5faa63bb-6207-4048-a6e3-d19d611aef45","Type":"ContainerStarted","Data":"9f078fcbe6e026d843f472b1be14c2cd021b643a71195fbdf5cefb101de1ad0e"}
Sep 16 15:53:11.904548 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:11.904512    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:11.905047 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:11.904577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:11.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:11.909259    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:12.050634 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:12.050597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc644567f-vpl46" event={"ID":"5faa63bb-6207-4048-a6e3-d19d611aef45","Type":"ContainerStarted","Data":"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"}
Sep 16 15:53:12.054620 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:12.054593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:12.072066 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:12.072012    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc644567f-vpl46" podStartSLOduration=2.071997254 podStartE2EDuration="2.071997254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:53:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:53:12.071303989 +0000 UTC m=+189.298761996" watchObservedRunningTime="2026-09-16 15:53:12.071997254 +0000 UTC m=+189.299455262"
Sep 16 15:53:20.730911 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:20.730867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:20.731333 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:20.730939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:20.735918 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:20.735897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:21.088102 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:21.088026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:53:21.133544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:21.133509    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:30.117129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:30.117048    2581 generic.go:356] "Generic (PLEG): container finished" podID="fa9bf1a4-1949-4346-bdb4-88795e371dd2" containerID="6714c61811a475cec98af670e10c0b1dea226e87c4689003d06394fa813c3f48" exitCode=0
Sep 16 15:53:30.117525 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:30.117124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw" event={"ID":"fa9bf1a4-1949-4346-bdb4-88795e371dd2","Type":"ContainerDied","Data":"6714c61811a475cec98af670e10c0b1dea226e87c4689003d06394fa813c3f48"}
Sep 16 15:53:30.117525 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:30.117497    2581 scope.go:117] "RemoveContainer" containerID="6714c61811a475cec98af670e10c0b1dea226e87c4689003d06394fa813c3f48"
Sep 16 15:53:31.122884 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:31.122851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-rcphw" event={"ID":"fa9bf1a4-1949-4346-bdb4-88795e371dd2","Type":"ContainerStarted","Data":"6facf89e9063af08c1858fee83381022c37990443b34db07c274687db452acf5"}
Sep 16 15:53:46.153996 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.153958    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c5f4f5db4-5rnql" podUID="1691c616-7a3d-4ca7-92aa-55403856f303" containerName="console" containerID="cri-o://687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c" gracePeriod=15
Sep 16 15:53:46.387082 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.387059    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:46.489023 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.488935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqs55\" (UniqueName: \"kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489023 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.488993    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489023 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489312 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489052    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489312 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489312 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489090    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert\") pod \"1691c616-7a3d-4ca7-92aa-55403856f303\" (UID: \"1691c616-7a3d-4ca7-92aa-55403856f303\") "
Sep 16 15:53:46.489564 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489536    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config" (OuterVolumeSpecName: "console-config") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:53:46.489633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489543    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca" (OuterVolumeSpecName: "service-ca") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:53:46.489633 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.489572    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:53:46.491420 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.491390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:53:46.491513 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.491418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55" (OuterVolumeSpecName: "kube-api-access-cqs55") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "kube-api-access-cqs55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:53:46.491513 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.491437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1691c616-7a3d-4ca7-92aa-55403856f303" (UID: "1691c616-7a3d-4ca7-92aa-55403856f303"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:53:46.590046 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590005    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqs55\" (UniqueName: \"kubernetes.io/projected/1691c616-7a3d-4ca7-92aa-55403856f303-kube-api-access-cqs55\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:46.590046 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590039    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-oauth-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:46.590046 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590051    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-console-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:46.590292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590059    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-service-ca\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:46.590292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590067    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1691c616-7a3d-4ca7-92aa-55403856f303-console-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:46.590292 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:46.590075    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1691c616-7a3d-4ca7-92aa-55403856f303-oauth-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:53:47.196352 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.196313    2581 generic.go:356] "Generic (PLEG): container finished" podID="1691c616-7a3d-4ca7-92aa-55403856f303" containerID="687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c" exitCode=0
Sep 16 15:53:47.196723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.196362    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f4f5db4-5rnql"
Sep 16 15:53:47.196723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.196361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f4f5db4-5rnql" event={"ID":"1691c616-7a3d-4ca7-92aa-55403856f303","Type":"ContainerDied","Data":"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"}
Sep 16 15:53:47.196723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.196487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f4f5db4-5rnql" event={"ID":"1691c616-7a3d-4ca7-92aa-55403856f303","Type":"ContainerDied","Data":"2dcf1f1ba604c2ef733025f9f9fd5d02b4048841b60546721e9d8c584d843ef6"}
Sep 16 15:53:47.196723 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.196517    2581 scope.go:117] "RemoveContainer" containerID="687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"
Sep 16 15:53:47.204952 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.204936    2581 scope.go:117] "RemoveContainer" containerID="687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"
Sep 16 15:53:47.205243 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:53:47.205221    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c\": container with ID starting with 687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c not found: ID does not exist" containerID="687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"
Sep 16 15:53:47.205339 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.205248    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c"} err="failed to get container status \"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c\": rpc error: code = NotFound desc = could not find container \"687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c\": container with ID starting with 687956c6f0c70e92df341fe14c0bb6e313a684d0407f527f77144e39175c825c not found: ID does not exist"
Sep 16 15:53:47.233444 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.233419    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:47.235626 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.235602    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5f4f5db4-5rnql"]
Sep 16 15:53:47.308711 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:53:47.308671    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1691c616-7a3d-4ca7-92aa-55403856f303" path="/var/lib/kubelet/pods/1691c616-7a3d-4ca7-92aa-55403856f303/volumes"
Sep 16 15:54:15.010322 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:15.010267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:54:15.012543 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:15.012522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af6df2ae-23ae-4e0f-bac7-c2b7806f29c9-metrics-certs\") pod \"network-metrics-daemon-rv47s\" (UID: \"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9\") " pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:54:15.107695 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:15.107662    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mmzn4\""
Sep 16 15:54:15.115956 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:15.115927    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rv47s"
Sep 16 15:54:15.386230 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:15.386192    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rv47s"]
Sep 16 15:54:15.388409 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:54:15.388379    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6df2ae_23ae_4e0f_bac7_c2b7806f29c9.slice/crio-0e38397553e46137b2daf19ef87bb18789c4adf1e287d26875d7e600a6b04fd6 WatchSource:0}: Error finding container 0e38397553e46137b2daf19ef87bb18789c4adf1e287d26875d7e600a6b04fd6: Status 404 returned error can't find the container with id 0e38397553e46137b2daf19ef87bb18789c4adf1e287d26875d7e600a6b04fd6
Sep 16 15:54:16.166764 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.166730    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:54:16.167202 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.167024    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1691c616-7a3d-4ca7-92aa-55403856f303" containerName="console"
Sep 16 15:54:16.167202 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.167037    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1691c616-7a3d-4ca7-92aa-55403856f303" containerName="console"
Sep 16 15:54:16.167202 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.167110    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1691c616-7a3d-4ca7-92aa-55403856f303" containerName="console"
Sep 16 15:54:16.171332 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.171306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.181654 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.181628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:54:16.222926 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.222890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.222969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.223000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.223020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.223043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.223077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m69p5\" (UniqueName: \"kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.223190 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.223108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.305447 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.305264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rv47s" event={"ID":"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9","Type":"ContainerStarted","Data":"0e38397553e46137b2daf19ef87bb18789c4adf1e287d26875d7e600a6b04fd6"}
Sep 16 15:54:16.323481 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323643 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323643 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323766 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323766 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m69p5\" (UniqueName: \"kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323766 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.323766 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.323730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.324214 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.324192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.324471 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.324447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.324571 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.324447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.324571 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.324497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.326704 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.326678    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.326807 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.326731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.333989 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.333969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m69p5\" (UniqueName: \"kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5\") pod \"console-6b54f56ccf-npr2j\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") " pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.482030 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.481943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:16.773780 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:16.773747    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:54:16.846214 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:54:16.846175    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3b795a_6471_4ae0_9388_0bc9923b3a49.slice/crio-93783e1207a5031648cbe153ee486540d099e34db5a7275e5a068ba54d9e575e WatchSource:0}: Error finding container 93783e1207a5031648cbe153ee486540d099e34db5a7275e5a068ba54d9e575e: Status 404 returned error can't find the container with id 93783e1207a5031648cbe153ee486540d099e34db5a7275e5a068ba54d9e575e
Sep 16 15:54:17.310440 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.310340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b54f56ccf-npr2j" event={"ID":"1a3b795a-6471-4ae0-9388-0bc9923b3a49","Type":"ContainerStarted","Data":"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"}
Sep 16 15:54:17.310440 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.310400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b54f56ccf-npr2j" event={"ID":"1a3b795a-6471-4ae0-9388-0bc9923b3a49","Type":"ContainerStarted","Data":"93783e1207a5031648cbe153ee486540d099e34db5a7275e5a068ba54d9e575e"}
Sep 16 15:54:17.312088 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.312061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rv47s" event={"ID":"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9","Type":"ContainerStarted","Data":"ce227cd355bd89182988bfd68968f60299ca0cdff7553a60eede89d0c647dfed"}
Sep 16 15:54:17.312216 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.312107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rv47s" event={"ID":"af6df2ae-23ae-4e0f-bac7-c2b7806f29c9","Type":"ContainerStarted","Data":"f9a167d68e0e18b3fc5dede287c5a940ee7fe52e632942915477a5d508275812"}
Sep 16 15:54:17.328646 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.328594    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b54f56ccf-npr2j" podStartSLOduration=1.328579076 podStartE2EDuration="1.328579076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:54:17.327135471 +0000 UTC m=+254.554593476" watchObservedRunningTime="2026-09-16 15:54:17.328579076 +0000 UTC m=+254.556037082"
Sep 16 15:54:17.342691 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:17.342634    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rv47s" podStartSLOduration=252.84134041 podStartE2EDuration="4m14.342617857s" totalImagesPullingTime="1.501277447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:54:17.341871402 +0000 UTC m=+254.569329420" watchObservedRunningTime="2026-09-16 15:54:17.342617857 +0000 UTC m=+254.570075863"
Sep 16 15:54:26.482604 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:26.482518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:26.482604 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:26.482594    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:26.487143 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:26.487123    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:27.351263 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:27.351231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:54:27.398268 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:27.398226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:54:52.417967 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.417926    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fc644567f-vpl46" podUID="5faa63bb-6207-4048-a6e3-d19d611aef45" containerName="console" containerID="cri-o://c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0" gracePeriod=15
Sep 16 15:54:52.668235 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.668169    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:54:52.710033 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710071    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710205 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710179    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl66d\" (UniqueName: \"kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d\") pod \"5faa63bb-6207-4048-a6e3-d19d611aef45\" (UID: \"5faa63bb-6207-4048-a6e3-d19d611aef45\") "
Sep 16 15:54:52.710762 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca" (OuterVolumeSpecName: "service-ca") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:54:52.710762 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:54:52.710762 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710611    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config" (OuterVolumeSpecName: "console-config") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:54:52.710762 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.710656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:54:52.712312 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.712288    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:54:52.712399 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.712320    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:54:52.712399 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.712325    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d" (OuterVolumeSpecName: "kube-api-access-sl66d") pod "5faa63bb-6207-4048-a6e3-d19d611aef45" (UID: "5faa63bb-6207-4048-a6e3-d19d611aef45"). InnerVolumeSpecName "kube-api-access-sl66d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:54:52.810778 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810744    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-service-ca\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810778 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810776    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-trusted-ca-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810984 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810789    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810984 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810803    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-console-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810984 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810815    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5faa63bb-6207-4048-a6e3-d19d611aef45-oauth-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810984 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810828    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5faa63bb-6207-4048-a6e3-d19d611aef45-console-oauth-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:52.810984 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:52.810840    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl66d\" (UniqueName: \"kubernetes.io/projected/5faa63bb-6207-4048-a6e3-d19d611aef45-kube-api-access-sl66d\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:54:53.448646 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.448608    2581 generic.go:356] "Generic (PLEG): container finished" podID="5faa63bb-6207-4048-a6e3-d19d611aef45" containerID="c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0" exitCode=0
Sep 16 15:54:53.449047 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.448656    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc644567f-vpl46"
Sep 16 15:54:53.449047 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.448673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc644567f-vpl46" event={"ID":"5faa63bb-6207-4048-a6e3-d19d611aef45","Type":"ContainerDied","Data":"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"}
Sep 16 15:54:53.449047 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.448722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc644567f-vpl46" event={"ID":"5faa63bb-6207-4048-a6e3-d19d611aef45","Type":"ContainerDied","Data":"9f078fcbe6e026d843f472b1be14c2cd021b643a71195fbdf5cefb101de1ad0e"}
Sep 16 15:54:53.449047 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.448749    2581 scope.go:117] "RemoveContainer" containerID="c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"
Sep 16 15:54:53.458388 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.458370    2581 scope.go:117] "RemoveContainer" containerID="c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"
Sep 16 15:54:53.458654 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:54:53.458632    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0\": container with ID starting with c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0 not found: ID does not exist" containerID="c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"
Sep 16 15:54:53.458751 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.458661    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0"} err="failed to get container status \"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0\": rpc error: code = NotFound desc = could not find container \"c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0\": container with ID starting with c904ec9a38dcc95b7d8ab3a9741258942044d2ca0c08c50f164660accc0045e0 not found: ID does not exist"
Sep 16 15:54:53.465896 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.465871    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:54:53.471429 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:53.471408    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fc644567f-vpl46"]
Sep 16 15:54:55.308645 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:54:55.308611    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5faa63bb-6207-4048-a6e3-d19d611aef45" path="/var/lib/kubelet/pods/5faa63bb-6207-4048-a6e3-d19d611aef45/volumes"
Sep 16 15:55:03.196573 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:03.196540    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 15:55:03.202106 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:03.202085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:55:03.207391 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:03.207371    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 15:55:03.210255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:03.210237    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 15:55:03.216040 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:03.216014    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 15:55:51.120681 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.120649    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:55:51.121086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.120944    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5faa63bb-6207-4048-a6e3-d19d611aef45" containerName="console"
Sep 16 15:55:51.121086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.120957    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5faa63bb-6207-4048-a6e3-d19d611aef45" containerName="console"
Sep 16 15:55:51.121086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.121018    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5faa63bb-6207-4048-a6e3-d19d611aef45" containerName="console"
Sep 16 15:55:51.124045 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.124008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.136955 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.136931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:55:51.251354 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251354 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.251622 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.251618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhlf\" (UniqueName: \"kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.352903 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.352870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.352903 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.352907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.352931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.352956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353129 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhlf\" (UniqueName: \"kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353685 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353809 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353894 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.353932 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.353902    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.355470 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.355438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.355564 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.355477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.361730 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.361708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhlf\" (UniqueName: \"kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf\") pod \"console-5d795c5599-4f7cp\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") " pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.433834 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.433797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:55:51.705018 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:51.704987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:55:51.707477 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:55:51.707446    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d53c26_ee66_4f50_abd3_0f381405ae8e.slice/crio-31d543b704666c135b3913ef5558e905d456aaa8bae276483536b58720e8b83c WatchSource:0}: Error finding container 31d543b704666c135b3913ef5558e905d456aaa8bae276483536b58720e8b83c: Status 404 returned error can't find the container with id 31d543b704666c135b3913ef5558e905d456aaa8bae276483536b58720e8b83c
Sep 16 15:55:52.664339 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:52.664299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d795c5599-4f7cp" event={"ID":"81d53c26-ee66-4f50-abd3-0f381405ae8e","Type":"ContainerStarted","Data":"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"}
Sep 16 15:55:52.664727 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:52.664363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d795c5599-4f7cp" event={"ID":"81d53c26-ee66-4f50-abd3-0f381405ae8e","Type":"ContainerStarted","Data":"31d543b704666c135b3913ef5558e905d456aaa8bae276483536b58720e8b83c"}
Sep 16 15:55:52.683794 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:55:52.683742    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d795c5599-4f7cp" podStartSLOduration=1.683726594 podStartE2EDuration="1.683726594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:55:52.682237831 +0000 UTC m=+349.909695839" watchObservedRunningTime="2026-09-16 15:55:52.683726594 +0000 UTC m=+349.911184599"
Sep 16 15:56:01.434308 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:01.434214    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:56:01.434308 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:01.434275    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:56:01.438968 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:01.438942    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:56:01.699818 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:01.699743    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:56:01.749412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:01.749382    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:56:26.772754 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:26.772695    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b54f56ccf-npr2j" podUID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" containerName="console" containerID="cri-o://15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38" gracePeriod=15
Sep 16 15:56:27.018835 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.018809    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:56:27.115041 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.114953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115041 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115084    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115110    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m69p5\" (UniqueName: \"kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115222    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115342 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config\") pod \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\" (UID: \"1a3b795a-6471-4ae0-9388-0bc9923b3a49\") "
Sep 16 15:56:27.115593 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:56:27.115648 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca" (OuterVolumeSpecName: "service-ca") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:56:27.115648 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:56:27.115715 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.115643    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config" (OuterVolumeSpecName: "console-config") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:56:27.117415 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.117386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:56:27.117516 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.117465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5" (OuterVolumeSpecName: "kube-api-access-m69p5") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "kube-api-access-m69p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:56:27.117516 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.117480    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1a3b795a-6471-4ae0-9388-0bc9923b3a49" (UID: "1a3b795a-6471-4ae0-9388-0bc9923b3a49"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:56:27.216261 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216227    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-service-ca\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216261 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216255    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m69p5\" (UniqueName: \"kubernetes.io/projected/1a3b795a-6471-4ae0-9388-0bc9923b3a49-kube-api-access-m69p5\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216261 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216265    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-oauth-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216274    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216283    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-trusted-ca-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216290    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3b795a-6471-4ae0-9388-0bc9923b3a49-oauth-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.216484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.216298    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3b795a-6471-4ae0-9388-0bc9923b3a49-console-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:56:27.795056 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.795021    2581 generic.go:356] "Generic (PLEG): container finished" podID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" containerID="15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38" exitCode=0
Sep 16 15:56:27.795466 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.795066    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b54f56ccf-npr2j"
Sep 16 15:56:27.795466 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.795094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b54f56ccf-npr2j" event={"ID":"1a3b795a-6471-4ae0-9388-0bc9923b3a49","Type":"ContainerDied","Data":"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"}
Sep 16 15:56:27.795466 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.795130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b54f56ccf-npr2j" event={"ID":"1a3b795a-6471-4ae0-9388-0bc9923b3a49","Type":"ContainerDied","Data":"93783e1207a5031648cbe153ee486540d099e34db5a7275e5a068ba54d9e575e"}
Sep 16 15:56:27.795466 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.795149    2581 scope.go:117] "RemoveContainer" containerID="15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"
Sep 16 15:56:27.802885 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.802866    2581 scope.go:117] "RemoveContainer" containerID="15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"
Sep 16 15:56:27.803190 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:56:27.803150    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38\": container with ID starting with 15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38 not found: ID does not exist" containerID="15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"
Sep 16 15:56:27.803302 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.803202    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38"} err="failed to get container status \"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38\": rpc error: code = NotFound desc = could not find container \"15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38\": container with ID starting with 15e941c64f207a1937809f2b7c05b6f4a6ea2149793fef204a7389a8c97cac38 not found: ID does not exist"
Sep 16 15:56:27.812637 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.812609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:56:27.817320 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:27.817299    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b54f56ccf-npr2j"]
Sep 16 15:56:29.308941 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:56:29.308910    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" path="/var/lib/kubelet/pods/1a3b795a-6471-4ae0-9388-0bc9923b3a49/volumes"
Sep 16 15:58:02.145641 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.145598    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-757dd6b8c8-l2fhw"]
Sep 16 15:58:02.146186 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.146003    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" containerName="console"
Sep 16 15:58:02.146186 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.146021    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" containerName="console"
Sep 16 15:58:02.146186 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.146121    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a3b795a-6471-4ae0-9388-0bc9923b3a49" containerName="console"
Sep 16 15:58:02.149296 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.149275    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.172009 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.171974    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-757dd6b8c8-l2fhw"]
Sep 16 15:58:02.227468 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzl7m\" (UniqueName: \"kubernetes.io/projected/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-kube-api-access-bzl7m\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227468 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-oauth-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227698 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227698 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-service-ca\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227698 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-trusted-ca-bundle\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227698 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-oauth-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.227698 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.227653    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328211 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl7m\" (UniqueName: \"kubernetes.io/projected/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-kube-api-access-bzl7m\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328211 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-oauth-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328432 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328432 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-service-ca\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328432 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-trusted-ca-bundle\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328577 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-oauth-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.328628 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.328585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.329125 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.329086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-oauth-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.329294 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.329254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-service-ca\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.329294 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.329261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-trusted-ca-bundle\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.329416 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.329321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.330850 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.330830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-serving-cert\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.330940 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.330901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-console-oauth-config\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.336546 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.336525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzl7m\" (UniqueName: \"kubernetes.io/projected/a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7-kube-api-access-bzl7m\") pod \"console-757dd6b8c8-l2fhw\" (UID: \"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7\") " pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.459019 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.458925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:02.735490 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:02.734611    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-757dd6b8c8-l2fhw"]
Sep 16 15:58:02.740730 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:58:02.740691    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b0cc5b_faf2_4cac_ab1a_4886fe01dab7.slice/crio-02b2210f618489e079db3ac343d2d13a884f50409b49bfb6f18b114ba579d514 WatchSource:0}: Error finding container 02b2210f618489e079db3ac343d2d13a884f50409b49bfb6f18b114ba579d514: Status 404 returned error can't find the container with id 02b2210f618489e079db3ac343d2d13a884f50409b49bfb6f18b114ba579d514
Sep 16 15:58:03.157302 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:03.157267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757dd6b8c8-l2fhw" event={"ID":"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7","Type":"ContainerStarted","Data":"3535c081a45e719e3a4b78f58882d8628bd1cfa6ca1e95c37c7fb2ebdf9fa08f"}
Sep 16 15:58:03.157690 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:03.157325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757dd6b8c8-l2fhw" event={"ID":"a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7","Type":"ContainerStarted","Data":"02b2210f618489e079db3ac343d2d13a884f50409b49bfb6f18b114ba579d514"}
Sep 16 15:58:03.177786 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:03.177731    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-757dd6b8c8-l2fhw" podStartSLOduration=1.17771429 podStartE2EDuration="1.17771429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:58:03.176361262 +0000 UTC m=+480.403819269" watchObservedRunningTime="2026-09-16 15:58:03.17771429 +0000 UTC m=+480.405172296"
Sep 16 15:58:12.459567 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:12.459518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:12.460217 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:12.459586    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:12.464569 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:12.464540    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:13.197700 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:13.197668    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-757dd6b8c8-l2fhw"
Sep 16 15:58:13.258873 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:13.258835    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:58:32.780002 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.779964    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"]
Sep 16 15:58:32.782568 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.782550    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.785368 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.785345    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kt8r9\""
Sep 16 15:58:32.785488 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.785347    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 15:58:32.786473 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.786426    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 15:58:32.791591 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.791567    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"]
Sep 16 15:58:32.875544 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.875503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwc79\" (UniqueName: \"kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.875725 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.875608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.875725 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.875663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.976861 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.976807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.977051 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.976872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.977051 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.976932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc79\" (UniqueName: \"kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.977214 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.977194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.977279 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.977237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:32.985526 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:32.985496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwc79\" (UniqueName: \"kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:33.092253 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:33.092150    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:33.362913 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:33.362880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"]
Sep 16 15:58:33.365500 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:58:33.365465    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc777e8fd_0a92_4eaa_aeec_a4e91391d395.slice/crio-0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500 WatchSource:0}: Error finding container 0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500: Status 404 returned error can't find the container with id 0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500
Sep 16 15:58:33.367341 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:33.367324    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 15:58:34.276311 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:34.276272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5" event={"ID":"c777e8fd-0a92-4eaa-aeec-a4e91391d395","Type":"ContainerStarted","Data":"0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500"}
Sep 16 15:58:38.279148 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.279073    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d795c5599-4f7cp" podUID="81d53c26-ee66-4f50-abd3-0f381405ae8e" containerName="console" containerID="cri-o://a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb" gracePeriod=15
Sep 16 15:58:38.813140 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.813116    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:58:38.930112 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930112 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930112 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930404 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930404 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930194    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930404 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930404 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqhlf\" (UniqueName: \"kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf\") pod \"81d53c26-ee66-4f50-abd3-0f381405ae8e\" (UID: \"81d53c26-ee66-4f50-abd3-0f381405ae8e\") "
Sep 16 15:58:38.930678 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config" (OuterVolumeSpecName: "console-config") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:58:38.930678 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930626    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:58:38.930678 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930631    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:58:38.930843 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.930668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca" (OuterVolumeSpecName: "service-ca") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 15:58:38.932219 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.932187    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:58:38.932391 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.932371    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf" (OuterVolumeSpecName: "kube-api-access-cqhlf") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "kube-api-access-cqhlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:58:38.932439 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:38.932387    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "81d53c26-ee66-4f50-abd3-0f381405ae8e" (UID: "81d53c26-ee66-4f50-abd3-0f381405ae8e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 15:58:39.031577 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031543    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031577 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031571    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-oauth-serving-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031577 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031582    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-service-ca\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031785 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031592    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqhlf\" (UniqueName: \"kubernetes.io/projected/81d53c26-ee66-4f50-abd3-0f381405ae8e-kube-api-access-cqhlf\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031785 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031600    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-oauth-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031785 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031633    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-console-config\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.031785 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.031642    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81d53c26-ee66-4f50-abd3-0f381405ae8e-trusted-ca-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:39.299956 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.299869    2581 generic.go:356] "Generic (PLEG): container finished" podID="81d53c26-ee66-4f50-abd3-0f381405ae8e" containerID="a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb" exitCode=0
Sep 16 15:58:39.299956 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.299924    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d795c5599-4f7cp"
Sep 16 15:58:39.300484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.299961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d795c5599-4f7cp" event={"ID":"81d53c26-ee66-4f50-abd3-0f381405ae8e","Type":"ContainerDied","Data":"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"}
Sep 16 15:58:39.300484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.300010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d795c5599-4f7cp" event={"ID":"81d53c26-ee66-4f50-abd3-0f381405ae8e","Type":"ContainerDied","Data":"31d543b704666c135b3913ef5558e905d456aaa8bae276483536b58720e8b83c"}
Sep 16 15:58:39.300484 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.300036    2581 scope.go:117] "RemoveContainer" containerID="a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"
Sep 16 15:58:39.301805 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.301770    2581 generic.go:356] "Generic (PLEG): container finished" podID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerID="04c58f9cf9d93036b34e72fb39f62f64304347c13fa7246d1a838559dc36518a" exitCode=0
Sep 16 15:58:39.301898 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.301844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5" event={"ID":"c777e8fd-0a92-4eaa-aeec-a4e91391d395","Type":"ContainerDied","Data":"04c58f9cf9d93036b34e72fb39f62f64304347c13fa7246d1a838559dc36518a"}
Sep 16 15:58:39.315740 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.315721    2581 scope.go:117] "RemoveContainer" containerID="a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"
Sep 16 15:58:39.315997 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:58:39.315977    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb\": container with ID starting with a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb not found: ID does not exist" containerID="a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"
Sep 16 15:58:39.316067 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.316001    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb"} err="failed to get container status \"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb\": rpc error: code = NotFound desc = could not find container \"a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb\": container with ID starting with a0b3bed9b85c27d42575eabdb0322315a8ecdeb9eed95bde0a3e5ac6daf8c0bb not found: ID does not exist"
Sep 16 15:58:39.335800 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.335769    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:58:39.339725 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:39.339702    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d795c5599-4f7cp"]
Sep 16 15:58:41.310790 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:41.310754    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81d53c26-ee66-4f50-abd3-0f381405ae8e" path="/var/lib/kubelet/pods/81d53c26-ee66-4f50-abd3-0f381405ae8e/volumes"
Sep 16 15:58:43.321647 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:43.321613    2581 generic.go:356] "Generic (PLEG): container finished" podID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerID="e299f33635a6618bb03e500ab46cd2b310115f124b00c9fe99e27fc6d10689b0" exitCode=0
Sep 16 15:58:43.322050 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:43.321679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5" event={"ID":"c777e8fd-0a92-4eaa-aeec-a4e91391d395","Type":"ContainerDied","Data":"e299f33635a6618bb03e500ab46cd2b310115f124b00c9fe99e27fc6d10689b0"}
Sep 16 15:58:50.355731 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:50.355696    2581 generic.go:356] "Generic (PLEG): container finished" podID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerID="4d3f0835117953c72c52085a1f52608e694358b2ed7c183af5c5a565d45c530d" exitCode=0
Sep 16 15:58:50.356113 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:50.355755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5" event={"ID":"c777e8fd-0a92-4eaa-aeec-a4e91391d395","Type":"ContainerDied","Data":"4d3f0835117953c72c52085a1f52608e694358b2ed7c183af5c5a565d45c530d"}
Sep 16 15:58:51.492812 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.492788    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:51.531330 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.531294    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwc79\" (UniqueName: \"kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79\") pod \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") "
Sep 16 15:58:51.531330 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.531337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util\") pod \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") "
Sep 16 15:58:51.531528 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.531360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle\") pod \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\" (UID: \"c777e8fd-0a92-4eaa-aeec-a4e91391d395\") "
Sep 16 15:58:51.531952 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.531929    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle" (OuterVolumeSpecName: "bundle") pod "c777e8fd-0a92-4eaa-aeec-a4e91391d395" (UID: "c777e8fd-0a92-4eaa-aeec-a4e91391d395"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 15:58:51.533454 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.533432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79" (OuterVolumeSpecName: "kube-api-access-kwc79") pod "c777e8fd-0a92-4eaa-aeec-a4e91391d395" (UID: "c777e8fd-0a92-4eaa-aeec-a4e91391d395"). InnerVolumeSpecName "kube-api-access-kwc79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:58:51.535621 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.535599    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util" (OuterVolumeSpecName: "util") pod "c777e8fd-0a92-4eaa-aeec-a4e91391d395" (UID: "c777e8fd-0a92-4eaa-aeec-a4e91391d395"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 15:58:51.631976 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.631949    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:51.631976 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.631976    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c777e8fd-0a92-4eaa-aeec-a4e91391d395-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:51.632179 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:51.631987    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwc79\" (UniqueName: \"kubernetes.io/projected/c777e8fd-0a92-4eaa-aeec-a4e91391d395-kube-api-access-kwc79\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:58:52.364874 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:52.364841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5" event={"ID":"c777e8fd-0a92-4eaa-aeec-a4e91391d395","Type":"ContainerDied","Data":"0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500"}
Sep 16 15:58:52.364874 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:52.364875    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1angrn5"
Sep 16 15:58:52.365077 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:52.364894    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e0dd7946d24cebe34532078ee344cc24b0094ef27ee8ae023f03595fbe14500"
Sep 16 15:58:55.123489 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123440    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"]
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123732    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="extract"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123744    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="extract"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123758    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="pull"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123763    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="pull"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123772    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d53c26-ee66-4f50-abd3-0f381405ae8e" containerName="console"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123778    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d53c26-ee66-4f50-abd3-0f381405ae8e" containerName="console"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123786    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="util"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123795    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="util"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123848    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d53c26-ee66-4f50-abd3-0f381405ae8e" containerName="console"
Sep 16 15:58:55.123856 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.123859    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c777e8fd-0a92-4eaa-aeec-a4e91391d395" containerName="extract"
Sep 16 15:58:55.166444 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.166409    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"]
Sep 16 15:58:55.166607 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.166506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.169477 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.169451    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 16 15:58:55.169477 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.169469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-zcgc6\""
Sep 16 15:58:55.169659 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.169490    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 16 15:58:55.169659 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.169490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 16 15:58:55.256175 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.256120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f2c82f4c-91cf-477d-8c04-3484fab567be-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.256346 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.256246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w772w\" (UniqueName: \"kubernetes.io/projected/f2c82f4c-91cf-477d-8c04-3484fab567be-kube-api-access-w772w\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.356952 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.356914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w772w\" (UniqueName: \"kubernetes.io/projected/f2c82f4c-91cf-477d-8c04-3484fab567be-kube-api-access-w772w\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.357251 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.357227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f2c82f4c-91cf-477d-8c04-3484fab567be-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.360408 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.360360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f2c82f4c-91cf-477d-8c04-3484fab567be-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.367594 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.367562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w772w\" (UniqueName: \"kubernetes.io/projected/f2c82f4c-91cf-477d-8c04-3484fab567be-kube-api-access-w772w\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-5v9k8\" (UID: \"f2c82f4c-91cf-477d-8c04-3484fab567be\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.476666 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.476632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:58:55.779332 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:55.779309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"]
Sep 16 15:58:55.781213 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:58:55.781136    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c82f4c_91cf_477d_8c04_3484fab567be.slice/crio-e8702b3bb7e99b5f1482c075a60519a5ea6568a796a7130d6cc66ed79e526c4c WatchSource:0}: Error finding container e8702b3bb7e99b5f1482c075a60519a5ea6568a796a7130d6cc66ed79e526c4c: Status 404 returned error can't find the container with id e8702b3bb7e99b5f1482c075a60519a5ea6568a796a7130d6cc66ed79e526c4c
Sep 16 15:58:56.383624 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:58:56.383586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8" event={"ID":"f2c82f4c-91cf-477d-8c04-3484fab567be","Type":"ContainerStarted","Data":"e8702b3bb7e99b5f1482c075a60519a5ea6568a796a7130d6cc66ed79e526c4c"}
Sep 16 15:59:00.402970 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.402936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8" event={"ID":"f2c82f4c-91cf-477d-8c04-3484fab567be","Type":"ContainerStarted","Data":"e502eb04d6929107a2ace6f2d8088d985678f01cc5954cb2944f5c276eea4522"}
Sep 16 15:59:00.403412 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.403049    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:59:00.425969 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.425920    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8" podStartSLOduration=1.579170742 podStartE2EDuration="5.425903351s" totalImagesPullingTime="3.846732609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:59:00.423842448 +0000 UTC m=+537.651300453" watchObservedRunningTime="2026-09-16 15:59:00.425903351 +0000 UTC m=+537.653361356"
Sep 16 15:59:00.456950 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.456910    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-pnq2g"]
Sep 16 15:59:00.460374 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.460351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.463255 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.463235    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 16 15:59:00.463350 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.463240    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-b4tx9\""
Sep 16 15:59:00.463514 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.463499    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 16 15:59:00.473424 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.473404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-pnq2g"]
Sep 16 15:59:00.501379 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.501346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt89f\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-kube-api-access-wt89f\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.501524 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.501396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-cabundle0\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.501524 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.501445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.602202 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.602151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt89f\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-kube-api-access-wt89f\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.602390 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.602211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-cabundle0\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.602390 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.602254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.602390 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:59:00.602344    2581 secret.go:281] references non-existent secret key: ca.crt
Sep 16 15:59:00.602390 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:59:00.602360    2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 16 15:59:00.602390 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:59:00.602370    2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-pnq2g: references non-existent secret key: ca.crt
Sep 16 15:59:00.602616 ip-10-0-137-111 kubenswrapper[2581]: E0916 15:59:00.602420    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates podName:b5fe3a4c-a85e-468b-b6c4-87747055dbaa nodeName:}" failed. No retries permitted until 2026-09-16 15:59:01.102405372 +0000 UTC m=+538.329863355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates") pod "keda-operator-579c8958f7-pnq2g" (UID: "b5fe3a4c-a85e-468b-b6c4-87747055dbaa") : references non-existent secret key: ca.crt
Sep 16 15:59:00.602960 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.602927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-cabundle0\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.613113 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.613086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt89f\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-kube-api-access-wt89f\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:00.693614 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.693539    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"]
Sep 16 15:59:00.696816 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.696800    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.699444 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.699422    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 16 15:59:00.708113 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.708088    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"]
Sep 16 15:59:00.803138 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.803097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sltnc\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-kube-api-access-sltnc\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.803337 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.803149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a34d1aea-4926-4b46-b5b5-57add06d4200-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.803337 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.803224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-certificates\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.904505 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.904465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sltnc\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-kube-api-access-sltnc\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.904677 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.904614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a34d1aea-4926-4b46-b5b5-57add06d4200-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.904677 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.904662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-certificates\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.905025 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.905005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a34d1aea-4926-4b46-b5b5-57add06d4200-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.907193 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.907144    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-certificates\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:00.912547 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:00.912519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sltnc\" (UniqueName: \"kubernetes.io/projected/a34d1aea-4926-4b46-b5b5-57add06d4200-kube-api-access-sltnc\") pod \"keda-metrics-apiserver-77857b7b98-rzpkw\" (UID: \"a34d1aea-4926-4b46-b5b5-57add06d4200\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:01.007708 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.007612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:01.106452 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.106417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:01.108851 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.108829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b5fe3a4c-a85e-468b-b6c4-87747055dbaa-certificates\") pod \"keda-operator-579c8958f7-pnq2g\" (UID: \"b5fe3a4c-a85e-468b-b6c4-87747055dbaa\") " pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:01.292195 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.292099    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"]
Sep 16 15:59:01.296121 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:59:01.296095    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34d1aea_4926_4b46_b5b5_57add06d4200.slice/crio-e22a399c890a6dd274cee9affb3258303da323efd88a63a3cb749315e9ff1f97 WatchSource:0}: Error finding container e22a399c890a6dd274cee9affb3258303da323efd88a63a3cb749315e9ff1f97: Status 404 returned error can't find the container with id e22a399c890a6dd274cee9affb3258303da323efd88a63a3cb749315e9ff1f97
Sep 16 15:59:01.371026 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.370985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:01.411310 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.411271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw" event={"ID":"a34d1aea-4926-4b46-b5b5-57add06d4200","Type":"ContainerStarted","Data":"e22a399c890a6dd274cee9affb3258303da323efd88a63a3cb749315e9ff1f97"}
Sep 16 15:59:01.659769 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:01.659742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-pnq2g"]
Sep 16 15:59:01.661783 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:59:01.661748    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fe3a4c_a85e_468b_b6c4_87747055dbaa.slice/crio-e3daddba228f7870c3e8c222713c5c30309b71e973a1c4164e48f988f202bea3 WatchSource:0}: Error finding container e3daddba228f7870c3e8c222713c5c30309b71e973a1c4164e48f988f202bea3: Status 404 returned error can't find the container with id e3daddba228f7870c3e8c222713c5c30309b71e973a1c4164e48f988f202bea3
Sep 16 15:59:02.417777 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:02.417736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-pnq2g" event={"ID":"b5fe3a4c-a85e-468b-b6c4-87747055dbaa","Type":"ContainerStarted","Data":"e3daddba228f7870c3e8c222713c5c30309b71e973a1c4164e48f988f202bea3"}
Sep 16 15:59:04.429606 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:04.429568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw" event={"ID":"a34d1aea-4926-4b46-b5b5-57add06d4200","Type":"ContainerStarted","Data":"f391d92081cf43360b73eed42b99823a994a742ec20feae1eb3c106046581350"}
Sep 16 15:59:04.430113 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:04.429819    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:04.447858 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:04.447804    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw" podStartSLOduration=1.804880437 podStartE2EDuration="4.447784916s" totalImagesPullingTime="2.642904479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:59:04.44590663 +0000 UTC m=+541.673364636" watchObservedRunningTime="2026-09-16 15:59:04.447784916 +0000 UTC m=+541.675242923"
Sep 16 15:59:07.448536 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:07.448496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-pnq2g" event={"ID":"b5fe3a4c-a85e-468b-b6c4-87747055dbaa","Type":"ContainerStarted","Data":"d6aa7a6dbacc464b14bb7dc06ede788e947cf2d41840d2c7abb7e8ba5e0167f6"}
Sep 16 15:59:07.448939 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:07.448699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:07.464482 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:07.464430    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-pnq2g" podStartSLOduration=2.601010278 podStartE2EDuration="7.46441381s" totalImagesPullingTime="4.863403532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 15:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 15:59:07.464367279 +0000 UTC m=+544.691825288" watchObservedRunningTime="2026-09-16 15:59:07.46441381 +0000 UTC m=+544.691871817"
Sep 16 15:59:15.439730 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:15.439695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-rzpkw"
Sep 16 15:59:21.413605 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:21.413573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-5v9k8"
Sep 16 15:59:28.455908 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:28.455876    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-pnq2g"
Sep 16 15:59:53.160593 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.160553    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"]
Sep 16 15:59:53.170945 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.170918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.173218 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.173188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"]
Sep 16 15:59:53.173632 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.173604    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kt8r9\""
Sep 16 15:59:53.173735 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.173634    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 15:59:53.174852 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.174831    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 15:59:53.237563 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.237532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.237735 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.237598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrf77\" (UniqueName: \"kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.237735 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.237632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.338901 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.338862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrf77\" (UniqueName: \"kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.339107 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.338908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.339107 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.338962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.339402 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.339380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.339469 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.339398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.347554 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.347528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrf77\" (UniqueName: \"kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.481662 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.481577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:53.750318 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:53.750293    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"]
Sep 16 15:59:53.752354 ip-10-0-137-111 kubenswrapper[2581]: W0916 15:59:53.752325    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2712d98b_4d28_4260_bad6_17d7f4345995.slice/crio-0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712 WatchSource:0}: Error finding container 0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712: Status 404 returned error can't find the container with id 0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712
Sep 16 15:59:54.651765 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:54.651731    2581 generic.go:356] "Generic (PLEG): container finished" podID="2712d98b-4d28-4260-bad6-17d7f4345995" containerID="83798ff9ffbb041ced191c0e5c84aa1a6254a853148b946c7a8007e0423d4a86" exitCode=0
Sep 16 15:59:54.652123 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:54.651808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6" event={"ID":"2712d98b-4d28-4260-bad6-17d7f4345995","Type":"ContainerDied","Data":"83798ff9ffbb041ced191c0e5c84aa1a6254a853148b946c7a8007e0423d4a86"}
Sep 16 15:59:54.652123 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:54.651860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6" event={"ID":"2712d98b-4d28-4260-bad6-17d7f4345995","Type":"ContainerStarted","Data":"0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712"}
Sep 16 15:59:57.668613 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:57.668580    2581 generic.go:356] "Generic (PLEG): container finished" podID="2712d98b-4d28-4260-bad6-17d7f4345995" containerID="7a7f7b5a873d0513118da7a28f9f313f01977ca5180e0fc57cf6f04e89e4cf3f" exitCode=0
Sep 16 15:59:57.668993 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:57.668650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6" event={"ID":"2712d98b-4d28-4260-bad6-17d7f4345995","Type":"ContainerDied","Data":"7a7f7b5a873d0513118da7a28f9f313f01977ca5180e0fc57cf6f04e89e4cf3f"}
Sep 16 15:59:58.675680 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:58.675639    2581 generic.go:356] "Generic (PLEG): container finished" podID="2712d98b-4d28-4260-bad6-17d7f4345995" containerID="0d5889dac3eb1dff317b2364f37240090dc804c6e741eaa0de77095b314106a9" exitCode=0
Sep 16 15:59:58.676089 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:58.675708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6" event={"ID":"2712d98b-4d28-4260-bad6-17d7f4345995","Type":"ContainerDied","Data":"0d5889dac3eb1dff317b2364f37240090dc804c6e741eaa0de77095b314106a9"}
Sep 16 15:59:59.813747 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.813722    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 15:59:59.893434 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.893394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrf77\" (UniqueName: \"kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77\") pod \"2712d98b-4d28-4260-bad6-17d7f4345995\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") "
Sep 16 15:59:59.893434 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.893440    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle\") pod \"2712d98b-4d28-4260-bad6-17d7f4345995\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") "
Sep 16 15:59:59.893693 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.893462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util\") pod \"2712d98b-4d28-4260-bad6-17d7f4345995\" (UID: \"2712d98b-4d28-4260-bad6-17d7f4345995\") "
Sep 16 15:59:59.894124 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.894097    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle" (OuterVolumeSpecName: "bundle") pod "2712d98b-4d28-4260-bad6-17d7f4345995" (UID: "2712d98b-4d28-4260-bad6-17d7f4345995"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 15:59:59.895512 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.895483    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77" (OuterVolumeSpecName: "kube-api-access-xrf77") pod "2712d98b-4d28-4260-bad6-17d7f4345995" (UID: "2712d98b-4d28-4260-bad6-17d7f4345995"). InnerVolumeSpecName "kube-api-access-xrf77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 15:59:59.899678 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.899651    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util" (OuterVolumeSpecName: "util") pod "2712d98b-4d28-4260-bad6-17d7f4345995" (UID: "2712d98b-4d28-4260-bad6-17d7f4345995"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 15:59:59.994086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.994001    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrf77\" (UniqueName: \"kubernetes.io/projected/2712d98b-4d28-4260-bad6-17d7f4345995-kube-api-access-xrf77\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:59:59.994086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.994028    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 15:59:59.994086 ip-10-0-137-111 kubenswrapper[2581]: I0916 15:59:59.994037    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2712d98b-4d28-4260-bad6-17d7f4345995-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:00:00.690427 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:00.690400    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6"
Sep 16 16:00:00.690571 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:00.690399    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406fgm6" event={"ID":"2712d98b-4d28-4260-bad6-17d7f4345995","Type":"ContainerDied","Data":"0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712"}
Sep 16 16:00:00.690571 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:00.690521    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f4893e8fdfc79c51e654af1b2700833806ddb7758a5f96e332757a3f1031712"
Sep 16 16:00:03.227439 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:03.227412    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:00:03.233179 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:03.233136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:00:03.236628 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:03.236606    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:00:03.242226 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:03.242208    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:00:06.030255 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030222    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"]
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="extract"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030532    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="extract"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030546    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="pull"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030552    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="pull"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030559    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="util"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030565    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="util"
Sep 16 16:00:06.030643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.030616    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2712d98b-4d28-4260-bad6-17d7f4345995" containerName="extract"
Sep 16 16:00:06.034679 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.034660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.037423 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.037400    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:00:06.037705 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.037461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kqjtz\""
Sep 16 16:00:06.037705 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.037461    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 16 16:00:06.047251 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.047224    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"]
Sep 16 16:00:06.144531 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.144489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca386be-ff5e-40bb-8bd4-0186251c7e06-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.144531 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.144535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb24j\" (UniqueName: \"kubernetes.io/projected/3ca386be-ff5e-40bb-8bd4-0186251c7e06-kube-api-access-kb24j\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.245024 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.244981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca386be-ff5e-40bb-8bd4-0186251c7e06-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.245248 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.245035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb24j\" (UniqueName: \"kubernetes.io/projected/3ca386be-ff5e-40bb-8bd4-0186251c7e06-kube-api-access-kb24j\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.245415 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.245396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ca386be-ff5e-40bb-8bd4-0186251c7e06-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.253498 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.253468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb24j\" (UniqueName: \"kubernetes.io/projected/3ca386be-ff5e-40bb-8bd4-0186251c7e06-kube-api-access-kb24j\") pod \"cert-manager-operator-controller-manager-684f79c9b9-dm2bp\" (UID: \"3ca386be-ff5e-40bb-8bd4-0186251c7e06\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.344275 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.344192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"
Sep 16 16:00:06.620936 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.620908    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp"]
Sep 16 16:00:06.623752 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:00:06.623723    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca386be_ff5e_40bb_8bd4_0186251c7e06.slice/crio-a38ce16ce91b6980547258c3fab39b281f7f9fa50ce2c52d48b5aac119ed2d93 WatchSource:0}: Error finding container a38ce16ce91b6980547258c3fab39b281f7f9fa50ce2c52d48b5aac119ed2d93: Status 404 returned error can't find the container with id a38ce16ce91b6980547258c3fab39b281f7f9fa50ce2c52d48b5aac119ed2d93
Sep 16 16:00:06.717422 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:06.717383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp" event={"ID":"3ca386be-ff5e-40bb-8bd4-0186251c7e06","Type":"ContainerStarted","Data":"a38ce16ce91b6980547258c3fab39b281f7f9fa50ce2c52d48b5aac119ed2d93"}
Sep 16 16:00:08.729630 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:08.729586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp" event={"ID":"3ca386be-ff5e-40bb-8bd4-0186251c7e06","Type":"ContainerStarted","Data":"8634946fec66e8a26808b67663f2aa456c8140c9309daac05b6d8c7a1744fdf7"}
Sep 16 16:00:08.752335 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:08.752269    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-dm2bp" podStartSLOduration=1.059913453 podStartE2EDuration="2.752250577s" totalImagesPullingTime="1.692337124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:00:08.750788529 +0000 UTC m=+605.978246536" watchObservedRunningTime="2026-09-16 16:00:08.752250577 +0000 UTC m=+605.979708585"
Sep 16 16:00:10.072876 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.072840    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8q25c"]
Sep 16 16:00:10.076247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.076225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.078909 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.078883    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 16:00:10.079017 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.078948    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-898pm\""
Sep 16 16:00:10.079017 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.078898    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 16:00:10.087400 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.087377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8q25c"]
Sep 16 16:00:10.179666 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.179620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.179808 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.179701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk5f5\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-kube-api-access-qk5f5\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.280640 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.280604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.280786 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.280670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk5f5\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-kube-api-access-qk5f5\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.290749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.290727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.291191 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.291149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk5f5\" (UniqueName: \"kubernetes.io/projected/fd76cf3e-8808-49db-af92-b4ead8490579-kube-api-access-qk5f5\") pod \"cert-manager-webhook-6486ffd997-8q25c\" (UID: \"fd76cf3e-8808-49db-af92-b4ead8490579\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.392313 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.392282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:10.662226 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.662109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8q25c"]
Sep 16 16:00:10.739130 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:10.739094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c" event={"ID":"fd76cf3e-8808-49db-af92-b4ead8490579","Type":"ContainerStarted","Data":"44edc5bf00e7367ccb448ac9de440b0ffed8febdef4af4065dce1b6ec6fa4229"}
Sep 16 16:00:14.760564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:14.760529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c" event={"ID":"fd76cf3e-8808-49db-af92-b4ead8490579","Type":"ContainerStarted","Data":"8e928ae83a499312fa23aa4732a23f2567c15afb26c65124d6fc306b68027388"}
Sep 16 16:00:14.760957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:14.760614    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:14.777501 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:14.777448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c" podStartSLOduration=1.279080584 podStartE2EDuration="4.777433897s" totalImagesPullingTime="3.498353313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:00:14.775463857 +0000 UTC m=+612.002921863" watchObservedRunningTime="2026-09-16 16:00:14.777433897 +0000 UTC m=+612.004891903"
Sep 16 16:00:15.716068 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.716034    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"]
Sep 16 16:00:15.719494 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.719477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.722391 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.722370    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:00:15.722526 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.722457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:00:15.722587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.722523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kt8r9\""
Sep 16 16:00:15.728085 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.728064    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"]
Sep 16 16:00:15.833184 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.833132    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8w7t\" (UniqueName: \"kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.833577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.833202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.833577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.833258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.934662 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.934632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w7t\" (UniqueName: \"kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.934782 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.934672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.934890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.934865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.935062 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.935046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.935174 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.935138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:15.943595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:15.943567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8w7t\" (UniqueName: \"kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:16.028744 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:16.028653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:16.300594 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:16.300568    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"]
Sep 16 16:00:16.303080 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:00:16.303049    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a8ff19_e60d_4f34_969a_f7743ea89f99.slice/crio-d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe WatchSource:0}: Error finding container d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe: Status 404 returned error can't find the container with id d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe
Sep 16 16:00:16.771193 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:16.771126    2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerID="fcc88f85fc45672a480c56797b812b9afaa68a204959d1620b69cd433e8610f6" exitCode=0
Sep 16 16:00:16.771334 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:16.771197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n" event={"ID":"88a8ff19-e60d-4f34-969a-f7743ea89f99","Type":"ContainerDied","Data":"fcc88f85fc45672a480c56797b812b9afaa68a204959d1620b69cd433e8610f6"}
Sep 16 16:00:16.771334 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:16.771260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n" event={"ID":"88a8ff19-e60d-4f34-969a-f7743ea89f99","Type":"ContainerStarted","Data":"d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe"}
Sep 16 16:00:19.786886 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:19.786847    2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerID="d418d342eee8dd315b3050424594516464e2275adfcb9708f156cec13c2f3ff8" exitCode=0
Sep 16 16:00:19.787285 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:19.786935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n" event={"ID":"88a8ff19-e60d-4f34-969a-f7743ea89f99","Type":"ContainerDied","Data":"d418d342eee8dd315b3050424594516464e2275adfcb9708f156cec13c2f3ff8"}
Sep 16 16:00:20.767669 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:20.767639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8q25c"
Sep 16 16:00:20.795063 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:20.795029    2581 generic.go:356] "Generic (PLEG): container finished" podID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerID="560cfe71b7dfd72f4201279fe00eb21d6e0cbc3dcf02f414f6b36b6459ed0247" exitCode=0
Sep 16 16:00:20.795457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:20.795113    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n" event={"ID":"88a8ff19-e60d-4f34-969a-f7743ea89f99","Type":"ContainerDied","Data":"560cfe71b7dfd72f4201279fe00eb21d6e0cbc3dcf02f414f6b36b6459ed0247"}
Sep 16 16:00:21.925900 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:21.925876    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:22.088153 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.088053    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8w7t\" (UniqueName: \"kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t\") pod \"88a8ff19-e60d-4f34-969a-f7743ea89f99\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") "
Sep 16 16:00:22.088153 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.088102    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle\") pod \"88a8ff19-e60d-4f34-969a-f7743ea89f99\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") "
Sep 16 16:00:22.088414 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.088224    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util\") pod \"88a8ff19-e60d-4f34-969a-f7743ea89f99\" (UID: \"88a8ff19-e60d-4f34-969a-f7743ea89f99\") "
Sep 16 16:00:22.088574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.088549    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle" (OuterVolumeSpecName: "bundle") pod "88a8ff19-e60d-4f34-969a-f7743ea89f99" (UID: "88a8ff19-e60d-4f34-969a-f7743ea89f99"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:00:22.090381 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.090348    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t" (OuterVolumeSpecName: "kube-api-access-n8w7t") pod "88a8ff19-e60d-4f34-969a-f7743ea89f99" (UID: "88a8ff19-e60d-4f34-969a-f7743ea89f99"). InnerVolumeSpecName "kube-api-access-n8w7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:00:22.092595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.092577    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util" (OuterVolumeSpecName: "util") pod "88a8ff19-e60d-4f34-969a-f7743ea89f99" (UID: "88a8ff19-e60d-4f34-969a-f7743ea89f99"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:00:22.189210 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.189129    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:00:22.189210 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.189204    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8w7t\" (UniqueName: \"kubernetes.io/projected/88a8ff19-e60d-4f34-969a-f7743ea89f99-kube-api-access-n8w7t\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:00:22.189210 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.189216    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a8ff19-e60d-4f34-969a-f7743ea89f99-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:00:22.806365 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.806336    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n"
Sep 16 16:00:22.806539 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.806371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frs84n" event={"ID":"88a8ff19-e60d-4f34-969a-f7743ea89f99","Type":"ContainerDied","Data":"d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe"}
Sep 16 16:00:22.806539 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:22.806413    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6f27257048c2ef82212deecb68d61bfd1e2f7b750205f9cc066218b466bd2fe"
Sep 16 16:00:29.335953 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.335875    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"]
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336200    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="pull"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336213    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="pull"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336224    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="util"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336229    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="util"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336237    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="extract"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336243    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="extract"
Sep 16 16:00:29.336457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.336290    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a8ff19-e60d-4f34-969a-f7743ea89f99" containerName="extract"
Sep 16 16:00:29.339096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.339079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.341789 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.341767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zhpmx\""
Sep 16 16:00:29.341900 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.341795    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:00:29.343241 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.343227    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 16:00:29.347374 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.347353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"]
Sep 16 16:00:29.454382 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.454337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9n9w\" (UniqueName: \"kubernetes.io/projected/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-kube-api-access-l9n9w\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.454569 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.454416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.554937 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.554898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9n9w\" (UniqueName: \"kubernetes.io/projected/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-kube-api-access-l9n9w\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.555129 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.554985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.555374 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.555355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.563300 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.563280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9n9w\" (UniqueName: \"kubernetes.io/projected/e3f2b8ea-adb3-44ec-b215-92fb259fc64b-kube-api-access-l9n9w\") pod \"openshift-lws-operator-fd8ccff4c-n9hpl\" (UID: \"e3f2b8ea-adb3-44ec-b215-92fb259fc64b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.649115 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.649079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"
Sep 16 16:00:29.921629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:29.921606    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl"]
Sep 16 16:00:29.924212 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:00:29.924179    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3f2b8ea_adb3_44ec_b215_92fb259fc64b.slice/crio-65385ef7b03300ce9497dcf3af6f100f2aca05f1a67adce89fdac4c36fdeb62a WatchSource:0}: Error finding container 65385ef7b03300ce9497dcf3af6f100f2aca05f1a67adce89fdac4c36fdeb62a: Status 404 returned error can't find the container with id 65385ef7b03300ce9497dcf3af6f100f2aca05f1a67adce89fdac4c36fdeb62a
Sep 16 16:00:30.844542 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:30.844503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl" event={"ID":"e3f2b8ea-adb3-44ec-b215-92fb259fc64b","Type":"ContainerStarted","Data":"65385ef7b03300ce9497dcf3af6f100f2aca05f1a67adce89fdac4c36fdeb62a"}
Sep 16 16:00:32.855938 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:32.855901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl" event={"ID":"e3f2b8ea-adb3-44ec-b215-92fb259fc64b","Type":"ContainerStarted","Data":"42b9380d36a41c3ab7a018e9cdeffdb6b7d6953f6790a77d6924fc02cc925e3c"}
Sep 16 16:00:32.873608 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:32.873556    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-n9hpl" podStartSLOduration=1.7346545789999999 podStartE2EDuration="3.873541371s" totalImagesPullingTime="2.138886792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:00:32.870580044 +0000 UTC m=+630.098038049" watchObservedRunningTime="2026-09-16 16:00:32.873541371 +0000 UTC m=+630.100999377"
Sep 16 16:00:42.871965 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.871927    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:00:42.875216 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.875199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.878007 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.877978    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 16:00:42.878129 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.878074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 16:00:42.878189 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.878143    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 16:00:42.883465 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.883446    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 16:00:42.884111 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.884082    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j2snt\""
Sep 16 16:00:42.884268 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.884255    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 16:00:42.884346 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.884330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 16:00:42.891621 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.891598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:00:42.963803 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.963803 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.964015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.964015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.964015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757rq\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.964015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:42.964015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:42.963961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065226 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757rq\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.065573 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.066001 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.065969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.068171 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.068126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.068274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.068149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.068274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.068218    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.068274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.068227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.073169 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.073122    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.073419 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.073397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757rq\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq\") pod \"istiod-openshift-gateway-7b7995d5cd-t9cb2\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.184767 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.184715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:43.462152 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.462123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:00:43.464253 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:00:43.464219    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d11aa76_2d37_40eb_a971_09e3684e91f9.slice/crio-eea71ef364c9a5dc55936e8e9ed7d11352ebd8318d8ac2a3cfc1442633413844 WatchSource:0}: Error finding container eea71ef364c9a5dc55936e8e9ed7d11352ebd8318d8ac2a3cfc1442633413844: Status 404 returned error can't find the container with id eea71ef364c9a5dc55936e8e9ed7d11352ebd8318d8ac2a3cfc1442633413844
Sep 16 16:00:43.907171 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:43.907120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" event={"ID":"4d11aa76-2d37-40eb-a971-09e3684e91f9","Type":"ContainerStarted","Data":"eea71ef364c9a5dc55936e8e9ed7d11352ebd8318d8ac2a3cfc1442633413844"}
Sep 16 16:00:48.705165 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:48.705116    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:00:48.705417 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:48.705206    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:00:48.934541 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:48.934497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" event={"ID":"4d11aa76-2d37-40eb-a971-09e3684e91f9","Type":"ContainerStarted","Data":"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"}
Sep 16 16:00:48.934689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:48.934610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:00:48.955291 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:48.955183    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" podStartSLOduration=1.716714203 podStartE2EDuration="6.955146586s" totalImagesPullingTime="5.238432383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:00:48.954053614 +0000 UTC m=+646.181511644" watchObservedRunningTime="2026-09-16 16:00:48.955146586 +0000 UTC m=+646.182604593"
Sep 16 16:00:49.941595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:00:49.941567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:01:02.083320 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.083276    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"]
Sep 16 16:01:02.087103 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.087078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.089847 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.089821    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:01:02.090899 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.090880    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kt8r9\""
Sep 16 16:01:02.091017 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.090882    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:01:02.095016 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.094991    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"]
Sep 16 16:01:02.178939 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.178903    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"]
Sep 16 16:01:02.182411 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.182392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.190469 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.190446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"]
Sep 16 16:01:02.231433 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.231433 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.231643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rxfp\" (UniqueName: \"kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.231643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.231643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdg79\" (UniqueName: \"kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.231643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.231633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.285330 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.285290    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"]
Sep 16 16:01:02.288809 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.288788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.296987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.296964    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"]
Sep 16 16:01:02.332365 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rxfp\" (UniqueName: \"kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.332563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.332563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdg79\" (UniqueName: \"kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.332563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.332563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.332563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.332846 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.332905 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.332905 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.332996 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.332934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.340535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.340475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdg79\" (UniqueName: \"kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.340535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.340485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rxfp\" (UniqueName: \"kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.391617 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.391583    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"]
Sep 16 16:01:02.395386 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.395365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.397227 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.397203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:02.402657 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.402627    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"]
Sep 16 16:01:02.433003 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.432968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtpc6\" (UniqueName: \"kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.433197 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.433014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.433197 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.433106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.492225 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.492191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:02.534520 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.534716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.534716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hznrf\" (UniqueName: \"kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.534716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.534716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.534944 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtpc6\" (UniqueName: \"kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.535002 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.534964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.535072 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.535045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.544727 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.544688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtpc6\" (UniqueName: \"kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.598499 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.598417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:02.636638 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.635985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.636638 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.636044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hznrf\" (UniqueName: \"kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.636638 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.636075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.636638 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.636509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.636965 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.636733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.645645 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.645400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hznrf\" (UniqueName: \"kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.677948 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.677916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"]
Sep 16 16:01:02.680433 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:01:02.680402    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3d14e8_e943_4028_89b2_b4b734a72dd9.slice/crio-0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053 WatchSource:0}: Error finding container 0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053: Status 404 returned error can't find the container with id 0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053
Sep 16 16:01:02.707428 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.707404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:02.763630 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.763566    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"]
Sep 16 16:01:02.765410 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:01:02.765343    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda955864a_bd2e_4f52_9823_c8977adc326f.slice/crio-6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa WatchSource:0}: Error finding container 6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa: Status 404 returned error can't find the container with id 6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa
Sep 16 16:01:02.871263 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.870788    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"]
Sep 16 16:01:02.987464 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:02.987430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"]
Sep 16 16:01:03.001845 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.001812    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerID="1af68a86aecec7b7dbc87283505a79b6dfe5090254711bb6b9fd4d54cbf389cf" exitCode=0
Sep 16 16:01:03.001982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.001895    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc" event={"ID":"9d3d14e8-e943-4028-89b2-b4b734a72dd9","Type":"ContainerDied","Data":"1af68a86aecec7b7dbc87283505a79b6dfe5090254711bb6b9fd4d54cbf389cf"}
Sep 16 16:01:03.001982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.001947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc" event={"ID":"9d3d14e8-e943-4028-89b2-b4b734a72dd9","Type":"ContainerStarted","Data":"0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053"}
Sep 16 16:01:03.003519 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.003493    2581 generic.go:356] "Generic (PLEG): container finished" podID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerID="85caede1332d1ce1d90886f99723d21543b85c107993da6a6b69f83b2265b268" exitCode=0
Sep 16 16:01:03.003632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.003576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g" event={"ID":"66f2294e-a7f9-4b2b-b036-690a883a15b6","Type":"ContainerDied","Data":"85caede1332d1ce1d90886f99723d21543b85c107993da6a6b69f83b2265b268"}
Sep 16 16:01:03.003632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.003624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g" event={"ID":"66f2294e-a7f9-4b2b-b036-690a883a15b6","Type":"ContainerStarted","Data":"48d76768ff8ebf190f4f672f45e364eb75402fc2183a8480fce577af160449e2"}
Sep 16 16:01:03.005259 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.005237    2581 generic.go:356] "Generic (PLEG): container finished" podID="a955864a-bd2e-4f52-9823-c8977adc326f" containerID="af86f99c00ee682b48afc911a65a1d75fead618dc2f06ddffd7e6463ceecef2a" exitCode=0
Sep 16 16:01:03.005370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.005301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j" event={"ID":"a955864a-bd2e-4f52-9823-c8977adc326f","Type":"ContainerDied","Data":"af86f99c00ee682b48afc911a65a1d75fead618dc2f06ddffd7e6463ceecef2a"}
Sep 16 16:01:03.005370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:03.005353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j" event={"ID":"a955864a-bd2e-4f52-9823-c8977adc326f","Type":"ContainerStarted","Data":"6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa"}
Sep 16 16:01:03.013944 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:01:03.013914    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95530336_99c2_40a8_86a7_a7ac19e14190.slice/crio-7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6 WatchSource:0}: Error finding container 7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6: Status 404 returned error can't find the container with id 7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6
Sep 16 16:01:04.012109 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:04.012067    2581 generic.go:356] "Generic (PLEG): container finished" podID="95530336-99c2-40a8-86a7-a7ac19e14190" containerID="37cc980d761cdf2cd085b854247f5620e48669c1f5e6b947dbdc3b5c7ccae7f1" exitCode=0
Sep 16 16:01:04.012511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:04.012150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4" event={"ID":"95530336-99c2-40a8-86a7-a7ac19e14190","Type":"ContainerDied","Data":"37cc980d761cdf2cd085b854247f5620e48669c1f5e6b947dbdc3b5c7ccae7f1"}
Sep 16 16:01:04.012511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:04.012218    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4" event={"ID":"95530336-99c2-40a8-86a7-a7ac19e14190","Type":"ContainerStarted","Data":"7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6"}
Sep 16 16:01:05.019714 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:05.019682    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerID="a0e27fb198fb1a4801f0d5fc841b17038144433f9bb6dddbb14792f7cc7b5d06" exitCode=0
Sep 16 16:01:05.020104 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:05.019747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc" event={"ID":"9d3d14e8-e943-4028-89b2-b4b734a72dd9","Type":"ContainerDied","Data":"a0e27fb198fb1a4801f0d5fc841b17038144433f9bb6dddbb14792f7cc7b5d06"}
Sep 16 16:01:06.027176 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:06.027124    2581 generic.go:356] "Generic (PLEG): container finished" podID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerID="2a1e9d9857bca9fdd1552318bde3389a6abe1026cdb1618e5ad002db29c0d386" exitCode=0
Sep 16 16:01:06.027589 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:06.027196    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc" event={"ID":"9d3d14e8-e943-4028-89b2-b4b734a72dd9","Type":"ContainerDied","Data":"2a1e9d9857bca9fdd1552318bde3389a6abe1026cdb1618e5ad002db29c0d386"}
Sep 16 16:01:07.034144 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.034103    2581 generic.go:356] "Generic (PLEG): container finished" podID="95530336-99c2-40a8-86a7-a7ac19e14190" containerID="99c7e97fa117e8a2e42d79324438e5555d017f6ca7689ca7ad78228e0daffdb4" exitCode=0
Sep 16 16:01:07.034621 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.034180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4" event={"ID":"95530336-99c2-40a8-86a7-a7ac19e14190","Type":"ContainerDied","Data":"99c7e97fa117e8a2e42d79324438e5555d017f6ca7689ca7ad78228e0daffdb4"}
Sep 16 16:01:07.235483 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.235455    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:07.378356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.378328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdg79\" (UniqueName: \"kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79\") pod \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") "
Sep 16 16:01:07.378482 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.378368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle\") pod \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") "
Sep 16 16:01:07.378482 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.378406    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util\") pod \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\" (UID: \"9d3d14e8-e943-4028-89b2-b4b734a72dd9\") "
Sep 16 16:01:07.378989 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.378964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle" (OuterVolumeSpecName: "bundle") pod "9d3d14e8-e943-4028-89b2-b4b734a72dd9" (UID: "9d3d14e8-e943-4028-89b2-b4b734a72dd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:07.380544 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.380518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79" (OuterVolumeSpecName: "kube-api-access-tdg79") pod "9d3d14e8-e943-4028-89b2-b4b734a72dd9" (UID: "9d3d14e8-e943-4028-89b2-b4b734a72dd9"). InnerVolumeSpecName "kube-api-access-tdg79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:01:07.384078 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.384040    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util" (OuterVolumeSpecName: "util") pod "9d3d14e8-e943-4028-89b2-b4b734a72dd9" (UID: "9d3d14e8-e943-4028-89b2-b4b734a72dd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:07.480096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.480054    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdg79\" (UniqueName: \"kubernetes.io/projected/9d3d14e8-e943-4028-89b2-b4b734a72dd9-kube-api-access-tdg79\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:07.480096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.480090    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:07.480096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:07.480099    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d3d14e8-e943-4028-89b2-b4b734a72dd9-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:08.041809 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.041773    2581 generic.go:356] "Generic (PLEG): container finished" podID="95530336-99c2-40a8-86a7-a7ac19e14190" containerID="563ce83393e2acf4425868de484d2a1c56e8cc9b90b4b04c6234cb681e3cac05" exitCode=0
Sep 16 16:01:08.042251 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.041857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4" event={"ID":"95530336-99c2-40a8-86a7-a7ac19e14190","Type":"ContainerDied","Data":"563ce83393e2acf4425868de484d2a1c56e8cc9b90b4b04c6234cb681e3cac05"}
Sep 16 16:01:08.043734 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.043708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc" event={"ID":"9d3d14e8-e943-4028-89b2-b4b734a72dd9","Type":"ContainerDied","Data":"0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053"}
Sep 16 16:01:08.043871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.043760    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fc5059aa78a3ce97139217ec81e236cd84c2f1f13b13240f3d573682445d053"
Sep 16 16:01:08.043871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.043715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6bhjc"
Sep 16 16:01:08.045549 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.045532    2581 generic.go:356] "Generic (PLEG): container finished" podID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerID="fd372b71a3577d1ff249c918f9454d24ff71a0f0c51e2324426c3149528fe774" exitCode=0
Sep 16 16:01:08.045647 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:08.045558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g" event={"ID":"66f2294e-a7f9-4b2b-b036-690a883a15b6","Type":"ContainerDied","Data":"fd372b71a3577d1ff249c918f9454d24ff71a0f0c51e2324426c3149528fe774"}
Sep 16 16:01:09.052594 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.052559    2581 generic.go:356] "Generic (PLEG): container finished" podID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerID="805ae8a9c1bba4600e998c34298260d982a3c3b339769e624831f30055015323" exitCode=0
Sep 16 16:01:09.052987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.052650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g" event={"ID":"66f2294e-a7f9-4b2b-b036-690a883a15b6","Type":"ContainerDied","Data":"805ae8a9c1bba4600e998c34298260d982a3c3b339769e624831f30055015323"}
Sep 16 16:01:09.180495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.180465    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:09.296415 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.296376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle\") pod \"95530336-99c2-40a8-86a7-a7ac19e14190\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") "
Sep 16 16:01:09.296589 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.296424    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util\") pod \"95530336-99c2-40a8-86a7-a7ac19e14190\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") "
Sep 16 16:01:09.296589 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.296458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hznrf\" (UniqueName: \"kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf\") pod \"95530336-99c2-40a8-86a7-a7ac19e14190\" (UID: \"95530336-99c2-40a8-86a7-a7ac19e14190\") "
Sep 16 16:01:09.296940 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.296913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle" (OuterVolumeSpecName: "bundle") pod "95530336-99c2-40a8-86a7-a7ac19e14190" (UID: "95530336-99c2-40a8-86a7-a7ac19e14190"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:09.298632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.298606    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf" (OuterVolumeSpecName: "kube-api-access-hznrf") pod "95530336-99c2-40a8-86a7-a7ac19e14190" (UID: "95530336-99c2-40a8-86a7-a7ac19e14190"). InnerVolumeSpecName "kube-api-access-hznrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:01:09.300751 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.300725    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util" (OuterVolumeSpecName: "util") pod "95530336-99c2-40a8-86a7-a7ac19e14190" (UID: "95530336-99c2-40a8-86a7-a7ac19e14190"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:09.397652 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.397610    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:09.397652 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.397645    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95530336-99c2-40a8-86a7-a7ac19e14190-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:09.397826 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:09.397678    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hznrf\" (UniqueName: \"kubernetes.io/projected/95530336-99c2-40a8-86a7-a7ac19e14190-kube-api-access-hznrf\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:10.059820 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.059788    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4"
Sep 16 16:01:10.060248 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.059811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7crwz4" event={"ID":"95530336-99c2-40a8-86a7-a7ac19e14190","Type":"ContainerDied","Data":"7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6"}
Sep 16 16:01:10.060248 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.059885    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d956c8b67d8a3b9217d2146da35e34b2a9147ad679a1971df5cdf87b22ecdb6"
Sep 16 16:01:10.189966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.189944    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:10.306972 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.306929    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util\") pod \"66f2294e-a7f9-4b2b-b036-690a883a15b6\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") "
Sep 16 16:01:10.307212 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.307069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle\") pod \"66f2294e-a7f9-4b2b-b036-690a883a15b6\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") "
Sep 16 16:01:10.307212 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.307118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtpc6\" (UniqueName: \"kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6\") pod \"66f2294e-a7f9-4b2b-b036-690a883a15b6\" (UID: \"66f2294e-a7f9-4b2b-b036-690a883a15b6\") "
Sep 16 16:01:10.307584 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.307556    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle" (OuterVolumeSpecName: "bundle") pod "66f2294e-a7f9-4b2b-b036-690a883a15b6" (UID: "66f2294e-a7f9-4b2b-b036-690a883a15b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:10.309348 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.309327    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6" (OuterVolumeSpecName: "kube-api-access-vtpc6") pod "66f2294e-a7f9-4b2b-b036-690a883a15b6" (UID: "66f2294e-a7f9-4b2b-b036-690a883a15b6"). InnerVolumeSpecName "kube-api-access-vtpc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:01:10.313382 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.313329    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util" (OuterVolumeSpecName: "util") pod "66f2294e-a7f9-4b2b-b036-690a883a15b6" (UID: "66f2294e-a7f9-4b2b-b036-690a883a15b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:10.408552 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.408515    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:10.408552 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.408546    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66f2294e-a7f9-4b2b-b036-690a883a15b6-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:10.408552 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:10.408558    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtpc6\" (UniqueName: \"kubernetes.io/projected/66f2294e-a7f9-4b2b-b036-690a883a15b6-kube-api-access-vtpc6\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:11.066637 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:11.066603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g" event={"ID":"66f2294e-a7f9-4b2b-b036-690a883a15b6","Type":"ContainerDied","Data":"48d76768ff8ebf190f4f672f45e364eb75402fc2183a8480fce577af160449e2"}
Sep 16 16:01:11.067015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:11.066651    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48d76768ff8ebf190f4f672f45e364eb75402fc2183a8480fce577af160449e2"
Sep 16 16:01:11.067015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:11.066654    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xvl4g"
Sep 16 16:01:19.107035 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:19.106999    2581 generic.go:356] "Generic (PLEG): container finished" podID="a955864a-bd2e-4f52-9823-c8977adc326f" containerID="d9e18f2e7b5c89e5c3e08567ef535e44b483268e6a96bc1d009be1aa3eb04f0c" exitCode=0
Sep 16 16:01:19.107445 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:19.107092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j" event={"ID":"a955864a-bd2e-4f52-9823-c8977adc326f","Type":"ContainerDied","Data":"d9e18f2e7b5c89e5c3e08567ef535e44b483268e6a96bc1d009be1aa3eb04f0c"}
Sep 16 16:01:20.113890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:20.113856    2581 generic.go:356] "Generic (PLEG): container finished" podID="a955864a-bd2e-4f52-9823-c8977adc326f" containerID="b09aab2e1fe93f3da6db96cad8f4489e30f5603ac3ccae50e85102249d525341" exitCode=0
Sep 16 16:01:20.114337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:20.113949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j" event={"ID":"a955864a-bd2e-4f52-9823-c8977adc326f","Type":"ContainerDied","Data":"b09aab2e1fe93f3da6db96cad8f4489e30f5603ac3ccae50e85102249d525341"}
Sep 16 16:01:21.253243 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.253219    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:21.401588 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.401554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util\") pod \"a955864a-bd2e-4f52-9823-c8977adc326f\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") "
Sep 16 16:01:21.401780 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.401623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle\") pod \"a955864a-bd2e-4f52-9823-c8977adc326f\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") "
Sep 16 16:01:21.401780 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.401743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rxfp\" (UniqueName: \"kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp\") pod \"a955864a-bd2e-4f52-9823-c8977adc326f\" (UID: \"a955864a-bd2e-4f52-9823-c8977adc326f\") "
Sep 16 16:01:21.402366 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.402337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle" (OuterVolumeSpecName: "bundle") pod "a955864a-bd2e-4f52-9823-c8977adc326f" (UID: "a955864a-bd2e-4f52-9823-c8977adc326f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:21.403844 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.403819    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp" (OuterVolumeSpecName: "kube-api-access-2rxfp") pod "a955864a-bd2e-4f52-9823-c8977adc326f" (UID: "a955864a-bd2e-4f52-9823-c8977adc326f"). InnerVolumeSpecName "kube-api-access-2rxfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:01:21.406052 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.406028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util" (OuterVolumeSpecName: "util") pod "a955864a-bd2e-4f52-9823-c8977adc326f" (UID: "a955864a-bd2e-4f52-9823-c8977adc326f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:01:21.503034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.503003    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rxfp\" (UniqueName: \"kubernetes.io/projected/a955864a-bd2e-4f52-9823-c8977adc326f-kube-api-access-2rxfp\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:21.503034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.503027    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-util\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:21.503034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:21.503036    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a955864a-bd2e-4f52-9823-c8977adc326f-bundle\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:01:22.126256 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:22.126222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j" event={"ID":"a955864a-bd2e-4f52-9823-c8977adc326f","Type":"ContainerDied","Data":"6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa"}
Sep 16 16:01:22.126256 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:22.126259    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avmw5j"
Sep 16 16:01:22.126463 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:22.126268    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6325b242eacbeeac588626e9a19addc0e8cc6f4927951655ca241eac994152aa"
Sep 16 16:01:29.908893 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.908854    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"]
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909204    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="extract"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909215    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="extract"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909224    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="pull"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909229    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="pull"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909239    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="util"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909245    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="util"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909251    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="extract"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909256    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="extract"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="util"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909271    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="util"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909280    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="extract"
Sep 16 16:01:29.909281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909286    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909292    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909297    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909305    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="util"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909310    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="util"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909318    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909322    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909331    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909335    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909340    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909344    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="pull"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909352    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="util"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909358    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="util"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909408    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3d14e8-e943-4028-89b2-b4b734a72dd9" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909416    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95530336-99c2-40a8-86a7-a7ac19e14190" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909422    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a955864a-bd2e-4f52-9823-c8977adc326f" containerName="extract"
Sep 16 16:01:29.909672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.909430    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f2294e-a7f9-4b2b-b036-690a883a15b6" containerName="extract"
Sep 16 16:01:29.912355 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.912338    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:29.915505 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.915407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 16:01:29.915505 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.915423    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 16:01:29.915672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.915524    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gh9tf\""
Sep 16 16:01:29.926188 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:29.926144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"]
Sep 16 16:01:30.080238 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.080192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.080380 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.080308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9t6\" (UniqueName: \"kubernetes.io/projected/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-kube-api-access-ll9t6\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.180853 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.180761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.181022 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.180861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9t6\" (UniqueName: \"kubernetes.io/projected/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-kube-api-access-ll9t6\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.181185 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.181141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.192823 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.192799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9t6\" (UniqueName: \"kubernetes.io/projected/fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb-kube-api-access-ll9t6\") pod \"kuadrant-operator-controller-manager-6d4cb79c-gbgm6\" (UID: \"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.223242 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.223215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:30.503118 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:30.503091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"]
Sep 16 16:01:30.504922 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:01:30.504895    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4bea24_1cfe_4069_bb22_3bffe0e3d9bb.slice/crio-4d8bfc0ae18e300b38d1a435f66c7c3e327f17839ddba4482f4b7c370c63d3f9 WatchSource:0}: Error finding container 4d8bfc0ae18e300b38d1a435f66c7c3e327f17839ddba4482f4b7c370c63d3f9: Status 404 returned error can't find the container with id 4d8bfc0ae18e300b38d1a435f66c7c3e327f17839ddba4482f4b7c370c63d3f9
Sep 16 16:01:31.174172 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:31.174121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6" event={"ID":"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb","Type":"ContainerStarted","Data":"4d8bfc0ae18e300b38d1a435f66c7c3e327f17839ddba4482f4b7c370c63d3f9"}
Sep 16 16:01:32.347464 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.347428    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"]
Sep 16 16:01:32.352046 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.352026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:32.354776 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.354757    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xwrqk\""
Sep 16 16:01:32.360013 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.359988    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"]
Sep 16 16:01:32.398689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.398650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cms5n\" (UniqueName: \"kubernetes.io/projected/832d05d3-8127-4771-a6bb-be10a99f6681-kube-api-access-cms5n\") pod \"authorino-operator-6fc8f864f4-q4fm2\" (UID: \"832d05d3-8127-4771-a6bb-be10a99f6681\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:32.499371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.499337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cms5n\" (UniqueName: \"kubernetes.io/projected/832d05d3-8127-4771-a6bb-be10a99f6681-kube-api-access-cms5n\") pod \"authorino-operator-6fc8f864f4-q4fm2\" (UID: \"832d05d3-8127-4771-a6bb-be10a99f6681\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:32.519824 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.519785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cms5n\" (UniqueName: \"kubernetes.io/projected/832d05d3-8127-4771-a6bb-be10a99f6681-kube-api-access-cms5n\") pod \"authorino-operator-6fc8f864f4-q4fm2\" (UID: \"832d05d3-8127-4771-a6bb-be10a99f6681\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:32.663924 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.663885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:32.958337 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:01:32.958297    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832d05d3_8127_4771_a6bb_be10a99f6681.slice/crio-1ac2fc6ac09578ea9244a1703235373907ae8b49aa58fbdcdd2a0435da7ec17b WatchSource:0}: Error finding container 1ac2fc6ac09578ea9244a1703235373907ae8b49aa58fbdcdd2a0435da7ec17b: Status 404 returned error can't find the container with id 1ac2fc6ac09578ea9244a1703235373907ae8b49aa58fbdcdd2a0435da7ec17b
Sep 16 16:01:32.958748 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:32.958725    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"]
Sep 16 16:01:33.186481 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:33.186430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2" event={"ID":"832d05d3-8127-4771-a6bb-be10a99f6681","Type":"ContainerStarted","Data":"1ac2fc6ac09578ea9244a1703235373907ae8b49aa58fbdcdd2a0435da7ec17b"}
Sep 16 16:01:37.211730 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.211690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6" event={"ID":"fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb","Type":"ContainerStarted","Data":"dcdacd66a1e1056f98663e6e2fd195cff0996aaf0ae697caf36bd97bd4b421d3"}
Sep 16 16:01:37.212152 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.211817    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:01:37.213434 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.213409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2" event={"ID":"832d05d3-8127-4771-a6bb-be10a99f6681","Type":"ContainerStarted","Data":"4b9c65c4c80a1bde784964cc605c44adbb430913d0ec6c0c8e7f03dbb116676e"}
Sep 16 16:01:37.213545 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.213469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:37.234913 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.234864    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6" podStartSLOduration=2.609134659 podStartE2EDuration="8.234852334s" totalImagesPullingTime="5.625717675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:01:37.234508865 +0000 UTC m=+694.461966870" watchObservedRunningTime="2026-09-16 16:01:37.234852334 +0000 UTC m=+694.462310338"
Sep 16 16:01:37.255205 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:37.255119    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2" podStartSLOduration=1.40299436 podStartE2EDuration="5.255102767s" totalImagesPullingTime="3.852108407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:01:37.254024152 +0000 UTC m=+694.481482159" watchObservedRunningTime="2026-09-16 16:01:37.255102767 +0000 UTC m=+694.482560772"
Sep 16 16:01:48.221655 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:48.221618    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-q4fm2"
Sep 16 16:01:48.222015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:01:48.221697    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6d4cb79c-gbgm6"
Sep 16 16:02:20.755863 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.755823    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:20.760585 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.760564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:20.763561 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.763540    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 16:02:20.763706 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.763612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-frpm4\""
Sep 16 16:02:20.767518 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.767492    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:20.851630 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.851597    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:20.909127 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.909089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvvc2\" (UniqueName: \"kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:20.909307 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:20.909192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.009748 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.009665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.009903 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.009768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvvc2\" (UniqueName: \"kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.010333 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.010313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.017724 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.017695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvvc2\" (UniqueName: \"kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2\") pod \"limitador-limitador-8588749977-zw9hj\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") " pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.071682 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.071650    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:21.344658 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.344631    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:21.346241 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:02:21.346212    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6219bf_4ccd_4cef_b22e_c0c4e5573fe2.slice/crio-79b3b16eb596ca71c0dc8fadd9ffade07cc7ed66edd8994af8fcfce0351164eb WatchSource:0}: Error finding container 79b3b16eb596ca71c0dc8fadd9ffade07cc7ed66edd8994af8fcfce0351164eb: Status 404 returned error can't find the container with id 79b3b16eb596ca71c0dc8fadd9ffade07cc7ed66edd8994af8fcfce0351164eb
Sep 16 16:02:21.438151 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:21.438116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" event={"ID":"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2","Type":"ContainerStarted","Data":"79b3b16eb596ca71c0dc8fadd9ffade07cc7ed66edd8994af8fcfce0351164eb"}
Sep 16 16:02:23.450329 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:23.450285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" event={"ID":"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2","Type":"ContainerStarted","Data":"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"}
Sep 16 16:02:23.450714 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:23.450350    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:23.468386 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:23.468342    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" podStartSLOduration=2.264503872 podStartE2EDuration="3.468328782s" totalImagesPullingTime="1.20382491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:02:23.466187241 +0000 UTC m=+740.693645247" watchObservedRunningTime="2026-09-16 16:02:23.468328782 +0000 UTC m=+740.695786825"
Sep 16 16:02:34.457053 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:34.457019    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:34.667356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:34.667312    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:34.667528 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:34.667494    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" podUID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" containerName="limitador" containerID="cri-o://20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75" gracePeriod=30
Sep 16 16:02:35.209463 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.209437    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:35.230513 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.230439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvvc2\" (UniqueName: \"kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2\") pod \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") "
Sep 16 16:02:35.230658 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.230513    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file\") pod \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\" (UID: \"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2\") "
Sep 16 16:02:35.230935 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.230903    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file" (OuterVolumeSpecName: "config-file") pod "8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" (UID: "8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:02:35.233235 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.233204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2" (OuterVolumeSpecName: "kube-api-access-lvvc2") pod "8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" (UID: "8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2"). InnerVolumeSpecName "kube-api-access-lvvc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:02:35.332346 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.332314    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-config-file\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:02:35.332474 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.332348    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvvc2\" (UniqueName: \"kubernetes.io/projected/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2-kube-api-access-lvvc2\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:02:35.518677 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.518589    2581 generic.go:356] "Generic (PLEG): container finished" podID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" containerID="20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75" exitCode=0
Sep 16 16:02:35.518677 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.518634    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj"
Sep 16 16:02:35.519093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.518676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" event={"ID":"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2","Type":"ContainerDied","Data":"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"}
Sep 16 16:02:35.519093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.518734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-zw9hj" event={"ID":"8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2","Type":"ContainerDied","Data":"79b3b16eb596ca71c0dc8fadd9ffade07cc7ed66edd8994af8fcfce0351164eb"}
Sep 16 16:02:35.519093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.518763    2581 scope.go:117] "RemoveContainer" containerID="20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"
Sep 16 16:02:35.527611 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.527577    2581 scope.go:117] "RemoveContainer" containerID="20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"
Sep 16 16:02:35.527860 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:02:35.527840    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75\": container with ID starting with 20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75 not found: ID does not exist" containerID="20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"
Sep 16 16:02:35.527936 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.527866    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75"} err="failed to get container status \"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75\": rpc error: code = NotFound desc = could not find container \"20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75\": container with ID starting with 20d2e415c3419b746042d55b03dda285c975c96857d54927b944473b1ec7aa75 not found: ID does not exist"
Sep 16 16:02:35.536254 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.536231    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:35.539575 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:35.539554    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-zw9hj"]
Sep 16 16:02:37.315118 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:37.315081    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" path="/var/lib/kubelet/pods/8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2/volumes"
Sep 16 16:02:43.148193 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.148148    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d9898bd99-ktrw6"]
Sep 16 16:02:43.148574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.148501    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" containerName="limitador"
Sep 16 16:02:43.148574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.148514    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" containerName="limitador"
Sep 16 16:02:43.148671 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.148583    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6219bf-4ccd-4cef-b22e-c0c4e5573fe2" containerName="limitador"
Sep 16 16:02:43.152743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.152723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.156398 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.156377    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 16:02:43.156514 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.156380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wrjb7\""
Sep 16 16:02:43.160534 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.160510    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d9898bd99-ktrw6"]
Sep 16 16:02:43.191813 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.187413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7003126-32d9-480a-b8ee-0de08daacbec-tls-cert\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.191813 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.187508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtsfz\" (UniqueName: \"kubernetes.io/projected/a7003126-32d9-480a-b8ee-0de08daacbec-kube-api-access-wtsfz\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.288267 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.288226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7003126-32d9-480a-b8ee-0de08daacbec-tls-cert\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.288431 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.288364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtsfz\" (UniqueName: \"kubernetes.io/projected/a7003126-32d9-480a-b8ee-0de08daacbec-kube-api-access-wtsfz\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.290659 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.290633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a7003126-32d9-480a-b8ee-0de08daacbec-tls-cert\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.296116 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.296091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtsfz\" (UniqueName: \"kubernetes.io/projected/a7003126-32d9-480a-b8ee-0de08daacbec-kube-api-access-wtsfz\") pod \"authorino-6d9898bd99-ktrw6\" (UID: \"a7003126-32d9-480a-b8ee-0de08daacbec\") " pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.463417 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.463337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d9898bd99-ktrw6"
Sep 16 16:02:43.731710 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:43.731681    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d9898bd99-ktrw6"]
Sep 16 16:02:43.733012 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:02:43.732983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7003126_32d9_480a_b8ee_0de08daacbec.slice/crio-b4976462fcd162329f471c3ae6d7d8ba985d532b64da4899ae98b463b0f1af48 WatchSource:0}: Error finding container b4976462fcd162329f471c3ae6d7d8ba985d532b64da4899ae98b463b0f1af48: Status 404 returned error can't find the container with id b4976462fcd162329f471c3ae6d7d8ba985d532b64da4899ae98b463b0f1af48
Sep 16 16:02:44.566621 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:44.566586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d9898bd99-ktrw6" event={"ID":"a7003126-32d9-480a-b8ee-0de08daacbec","Type":"ContainerStarted","Data":"b4976462fcd162329f471c3ae6d7d8ba985d532b64da4899ae98b463b0f1af48"}
Sep 16 16:02:46.580062 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:46.580018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d9898bd99-ktrw6" event={"ID":"a7003126-32d9-480a-b8ee-0de08daacbec","Type":"ContainerStarted","Data":"3a6aa239ba3275ab94d797ce58e35dc5d7d4dd36436ca0276d8de0bd001fc21e"}
Sep 16 16:02:46.595530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:02:46.595484    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d9898bd99-ktrw6" podStartSLOduration=1.239222078 podStartE2EDuration="3.595468637s" totalImagesPullingTime="2.356246559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:02:46.593574664 +0000 UTC m=+763.821032755" watchObservedRunningTime="2026-09-16 16:02:46.595468637 +0000 UTC m=+763.822926643"
Sep 16 16:03:02.818850 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.818813    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"]
Sep 16 16:03:02.822191 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.822152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.832944 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.832916    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"]
Sep 16 16:03:02.957719 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957719 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02538c77-74c8-4198-816f-d3aca553961e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xnh\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-kube-api-access-g2xnh\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:02.957966 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:02.957922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.058899 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.058856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.058910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059199 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02538c77-74c8-4198-816f-d3aca553961e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059199 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059322 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xnh\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-kube-api-access-g2xnh\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059322 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.059936 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.059911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.061371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.061339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.061463 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.061399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.061710 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.061690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02538c77-74c8-4198-816f-d3aca553961e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.061755 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.061720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02538c77-74c8-4198-816f-d3aca553961e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.070982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.070936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.071482 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.071463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xnh\" (UniqueName: \"kubernetes.io/projected/02538c77-74c8-4198-816f-d3aca553961e-kube-api-access-g2xnh\") pod \"istiod-openshift-gateway-5677ccb49b-fddwk\" (UID: \"02538c77-74c8-4198-816f-d3aca553961e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.131211 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.131186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.409794 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.409767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"]
Sep 16 16:03:03.411421 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:03:03.411389    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02538c77_74c8_4198_816f_d3aca553961e.slice/crio-fec796aaa294ff27bea997c8ef38111d2c41d5cd880414e764dca6f835dffbe3 WatchSource:0}: Error finding container fec796aaa294ff27bea997c8ef38111d2c41d5cd880414e764dca6f835dffbe3: Status 404 returned error can't find the container with id fec796aaa294ff27bea997c8ef38111d2c41d5cd880414e764dca6f835dffbe3
Sep 16 16:03:03.413564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.413531    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:03:03.413627 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.413605    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:03:03.675040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.674935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk" event={"ID":"02538c77-74c8-4198-816f-d3aca553961e","Type":"ContainerStarted","Data":"fda2f8a4459128d50aa7a0945ae4c40a7f3184150f0f889a133b00631268673b"}
Sep 16 16:03:03.675040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.674994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk" event={"ID":"02538c77-74c8-4198-816f-d3aca553961e","Type":"ContainerStarted","Data":"fec796aaa294ff27bea997c8ef38111d2c41d5cd880414e764dca6f835dffbe3"}
Sep 16 16:03:03.675040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.675020    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:03.695911 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:03.695859    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk" podStartSLOduration=1.6958421860000001 podStartE2EDuration="1.695842186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:03:03.694046782 +0000 UTC m=+780.921504790" watchObservedRunningTime="2026-09-16 16:03:03.695842186 +0000 UTC m=+780.923300191"
Sep 16 16:03:04.681996 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:04.681964    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fddwk"
Sep 16 16:03:04.737514 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:04.737472    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:03:04.737716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:04.737690    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerName="discovery" containerID="cri-o://02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02" gracePeriod=30
Sep 16 16:03:04.986940 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:04.986918    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:03:05.077299 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077267    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077299 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077301    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077543 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077543 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077460    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077657 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077657 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077631    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.077756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-757rq\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq\") pod \"4d11aa76-2d37-40eb-a971-09e3684e91f9\" (UID: \"4d11aa76-2d37-40eb-a971-09e3684e91f9\") "
Sep 16 16:03:05.078081 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.077994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:03:05.079840 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.079803    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:03:05.079963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.079840    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:03:05.080035 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.079959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts" (OuterVolumeSpecName: "cacerts") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:03:05.080077 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.080060    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs" (OuterVolumeSpecName: "local-certs") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:03:05.080112 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.080077    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:03:05.080149 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.080105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq" (OuterVolumeSpecName: "kube-api-access-757rq") pod "4d11aa76-2d37-40eb-a971-09e3684e91f9" (UID: "4d11aa76-2d37-40eb-a971-09e3684e91f9"). InnerVolumeSpecName "kube-api-access-757rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:03:05.178759 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178727    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-dns-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178759 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178755    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-csr-ca-configmap\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178759 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178765    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-kubeconfig\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178980 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178773    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-757rq\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-kube-api-access-757rq\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178980 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178783    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d11aa76-2d37-40eb-a971-09e3684e91f9-istio-token\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178980 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178792    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4d11aa76-2d37-40eb-a971-09e3684e91f9-cacerts\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.178980 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.178799    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d11aa76-2d37-40eb-a971-09e3684e91f9-local-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:05.686370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.686326    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerID="02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02" exitCode=0
Sep 16 16:03:05.686851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.686378    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"
Sep 16 16:03:05.686851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.686418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" event={"ID":"4d11aa76-2d37-40eb-a971-09e3684e91f9","Type":"ContainerDied","Data":"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"}
Sep 16 16:03:05.686851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.686470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" event={"ID":"4d11aa76-2d37-40eb-a971-09e3684e91f9","Type":"ContainerDied","Data":"eea71ef364c9a5dc55936e8e9ed7d11352ebd8318d8ac2a3cfc1442633413844"}
Sep 16 16:03:05.686851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.686491    2581 scope.go:117] "RemoveContainer" containerID="02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"
Sep 16 16:03:05.695102 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.695082    2581 scope.go:117] "RemoveContainer" containerID="02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"
Sep 16 16:03:05.695384 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:03:05.695362    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02\": container with ID starting with 02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02 not found: ID does not exist" containerID="02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"
Sep 16 16:03:05.695447 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.695393    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02"} err="failed to get container status \"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02\": rpc error: code = NotFound desc = could not find container \"02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02\": container with ID starting with 02e5bb2711f65bbe8491f43890bf45162320c8edf0f0c98a757c322dd76a8c02 not found: ID does not exist"
Sep 16 16:03:05.707881 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.707853    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:03:05.711626 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:05.711604    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2"]
Sep 16 16:03:07.310308 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:07.310274    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" path="/var/lib/kubelet/pods/4d11aa76-2d37-40eb-a971-09e3684e91f9/volumes"
Sep 16 16:03:09.940822 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:09.940769    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-t9cb2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 16:03:09.941307 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:09.940843    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t9cb2" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.35:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 16:03:14.061319 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.061276    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:14.061740 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.061715    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerName="discovery"
Sep 16 16:03:14.061740 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.061733    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerName="discovery"
Sep 16 16:03:14.061811 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.061807    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d11aa76-2d37-40eb-a971-09e3684e91f9" containerName="discovery"
Sep 16 16:03:14.065996 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.065976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.068640 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.068615    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 16 16:03:14.068753 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.068656    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 16 16:03:14.068753 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.068658    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 16 16:03:14.069853 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.069839    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-flj8m\""
Sep 16 16:03:14.073069 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.073046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:14.156482 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.156449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.156648 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.156521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66mt\" (UniqueName: \"kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.257663 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.257630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66mt\" (UniqueName: \"kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.257837 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.257690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.257837 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:03:14.257793    2581 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 16 16:03:14.257930 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:03:14.257858    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert podName:4be931ed-dab2-4acc-b03e-9e59eda9c10a nodeName:}" failed. No retries permitted until 2026-09-16 16:03:14.757842311 +0000 UTC m=+791.985300295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert") pod "kserve-controller-manager-74fc95bbfc-hj697" (UID: "4be931ed-dab2-4acc-b03e-9e59eda9c10a") : secret "kserve-webhook-server-cert" not found
Sep 16 16:03:14.265921 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.265899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66mt\" (UniqueName: \"kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.762811 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.762775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.765151 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.765133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") pod \"kserve-controller-manager-74fc95bbfc-hj697\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") " pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:14.977716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:14.977680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:15.248665 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:15.248637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:15.251030 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:03:15.251005    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be931ed_dab2_4acc_b03e_9e59eda9c10a.slice/crio-550b88c05da46d1fe4a199f47cbfe8a3176b9bed7e064c1551588279a67ceb54 WatchSource:0}: Error finding container 550b88c05da46d1fe4a199f47cbfe8a3176b9bed7e064c1551588279a67ceb54: Status 404 returned error can't find the container with id 550b88c05da46d1fe4a199f47cbfe8a3176b9bed7e064c1551588279a67ceb54
Sep 16 16:03:15.744587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:15.744548    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" event={"ID":"4be931ed-dab2-4acc-b03e-9e59eda9c10a","Type":"ContainerStarted","Data":"550b88c05da46d1fe4a199f47cbfe8a3176b9bed7e064c1551588279a67ceb54"}
Sep 16 16:03:18.764025 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:18.763986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" event={"ID":"4be931ed-dab2-4acc-b03e-9e59eda9c10a","Type":"ContainerStarted","Data":"745e67e1cb4abc885bc64afc9087d89b53a770e3f23357becc799dbc27e0841f"}
Sep 16 16:03:18.764440 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:18.764115    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:18.780271 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:18.780224    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" podStartSLOduration=1.764473242 podStartE2EDuration="4.780208715s" totalImagesPullingTime="3.015735473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:03:18.779045314 +0000 UTC m=+796.006503324" watchObservedRunningTime="2026-09-16 16:03:18.780208715 +0000 UTC m=+796.007666720"
Sep 16 16:03:49.774433 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:49.774396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:57.833509 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.833463    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:57.833925 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.833678    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" podUID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" containerName="manager" containerID="cri-o://745e67e1cb4abc885bc64afc9087d89b53a770e3f23357becc799dbc27e0841f" gracePeriod=10
Sep 16 16:03:57.858637 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.858601    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-bphgb"]
Sep 16 16:03:57.862537 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.862513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:57.871502 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.871475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-bphgb"]
Sep 16 16:03:57.938877 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.938834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a116d94b-fe7f-4fd1-b458-42e0167762ad-cert\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:57.939043 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.938943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzjg\" (UniqueName: \"kubernetes.io/projected/a116d94b-fe7f-4fd1-b458-42e0167762ad-kube-api-access-zdzjg\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:57.975834 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.975799    2581 generic.go:356] "Generic (PLEG): container finished" podID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" containerID="745e67e1cb4abc885bc64afc9087d89b53a770e3f23357becc799dbc27e0841f" exitCode=0
Sep 16 16:03:57.976003 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:57.975870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" event={"ID":"4be931ed-dab2-4acc-b03e-9e59eda9c10a","Type":"ContainerDied","Data":"745e67e1cb4abc885bc64afc9087d89b53a770e3f23357becc799dbc27e0841f"}
Sep 16 16:03:58.039445 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.039410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzjg\" (UniqueName: \"kubernetes.io/projected/a116d94b-fe7f-4fd1-b458-42e0167762ad-kube-api-access-zdzjg\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:58.039624 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.039474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a116d94b-fe7f-4fd1-b458-42e0167762ad-cert\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:58.041742 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.041717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a116d94b-fe7f-4fd1-b458-42e0167762ad-cert\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:58.047753 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.047724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzjg\" (UniqueName: \"kubernetes.io/projected/a116d94b-fe7f-4fd1-b458-42e0167762ad-kube-api-access-zdzjg\") pod \"kserve-controller-manager-74fc95bbfc-bphgb\" (UID: \"a116d94b-fe7f-4fd1-b458-42e0167762ad\") " pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:58.084655 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.084596    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:58.140140 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.140106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") pod \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") "
Sep 16 16:03:58.140346 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.140208    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b66mt\" (UniqueName: \"kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt\") pod \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\" (UID: \"4be931ed-dab2-4acc-b03e-9e59eda9c10a\") "
Sep 16 16:03:58.142413 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.142373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert" (OuterVolumeSpecName: "cert") pod "4be931ed-dab2-4acc-b03e-9e59eda9c10a" (UID: "4be931ed-dab2-4acc-b03e-9e59eda9c10a"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:03:58.142528 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.142492    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt" (OuterVolumeSpecName: "kube-api-access-b66mt") pod "4be931ed-dab2-4acc-b03e-9e59eda9c10a" (UID: "4be931ed-dab2-4acc-b03e-9e59eda9c10a"). InnerVolumeSpecName "kube-api-access-b66mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:03:58.212071 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.212025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:03:58.241139 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.241096    2581 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4be931ed-dab2-4acc-b03e-9e59eda9c10a-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:58.241139 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.241127    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b66mt\" (UniqueName: \"kubernetes.io/projected/4be931ed-dab2-4acc-b03e-9e59eda9c10a-kube-api-access-b66mt\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:03:58.484039 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.484011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-bphgb"]
Sep 16 16:03:58.485652 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:03:58.485616    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda116d94b_fe7f_4fd1_b458_42e0167762ad.slice/crio-48f7b32048bad2d2cd8791df257978350892e27df755ba37993022d11b364b5d WatchSource:0}: Error finding container 48f7b32048bad2d2cd8791df257978350892e27df755ba37993022d11b364b5d: Status 404 returned error can't find the container with id 48f7b32048bad2d2cd8791df257978350892e27df755ba37993022d11b364b5d
Sep 16 16:03:58.486954 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.486936    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:03:58.982614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.982582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697"
Sep 16 16:03:58.983030 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.982580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-hj697" event={"ID":"4be931ed-dab2-4acc-b03e-9e59eda9c10a","Type":"ContainerDied","Data":"550b88c05da46d1fe4a199f47cbfe8a3176b9bed7e064c1551588279a67ceb54"}
Sep 16 16:03:58.983030 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.982724    2581 scope.go:117] "RemoveContainer" containerID="745e67e1cb4abc885bc64afc9087d89b53a770e3f23357becc799dbc27e0841f"
Sep 16 16:03:58.984108 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:58.983875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb" event={"ID":"a116d94b-fe7f-4fd1-b458-42e0167762ad","Type":"ContainerStarted","Data":"48f7b32048bad2d2cd8791df257978350892e27df755ba37993022d11b364b5d"}
Sep 16 16:03:59.005047 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:59.005018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:59.012224 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:59.012188    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-74fc95bbfc-hj697"]
Sep 16 16:03:59.311071 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:03:59.311037    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" path="/var/lib/kubelet/pods/4be931ed-dab2-4acc-b03e-9e59eda9c10a/volumes"
Sep 16 16:04:02.002594 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:02.002556    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb" event={"ID":"a116d94b-fe7f-4fd1-b458-42e0167762ad","Type":"ContainerStarted","Data":"28d1f06c0a09bcc9492f510244325c1f9d5ca31d12f4e8888510a8c589147d33"}
Sep 16 16:04:02.002973 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:02.002638    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:04:33.013872 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:33.013836    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb"
Sep 16 16:04:33.034176 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:33.034109    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-74fc95bbfc-bphgb" podStartSLOduration=33.12165686 podStartE2EDuration="36.034094724s" totalImagesPullingTime="2.912437864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:04:02.033760112 +0000 UTC m=+839.261218121" watchObservedRunningTime="2026-09-16 16:04:33.034094724 +0000 UTC m=+870.261552729"
Sep 16 16:04:34.334289 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.334246    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-kvq8k"]
Sep 16 16:04:34.334669 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.334613    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" containerName="manager"
Sep 16 16:04:34.334669 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.334625    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" containerName="manager"
Sep 16 16:04:34.334744 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.334684    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4be931ed-dab2-4acc-b03e-9e59eda9c10a" containerName="manager"
Sep 16 16:04:34.337562 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.337539    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.340233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.340204    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 16 16:04:34.340370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.340235    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-wbbx6\""
Sep 16 16:04:34.340370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.340204    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 16 16:04:34.346009 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.345974    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-kvq8k"]
Sep 16 16:04:34.454104 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.454069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gxpr\" (UniqueName: \"kubernetes.io/projected/95edd79f-f65d-4a62-bcbf-bef82c5ea794-kube-api-access-2gxpr\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.454308 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.454232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-metrics-certs\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.454308 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.454299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-cert\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.555035 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.554993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gxpr\" (UniqueName: \"kubernetes.io/projected/95edd79f-f65d-4a62-bcbf-bef82c5ea794-kube-api-access-2gxpr\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.555239 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.555067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-metrics-certs\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.555239 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.555106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-cert\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.557580 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.557550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-metrics-certs\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.557691 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.557591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95edd79f-f65d-4a62-bcbf-bef82c5ea794-cert\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.585694 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.585611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gxpr\" (UniqueName: \"kubernetes.io/projected/95edd79f-f65d-4a62-bcbf-bef82c5ea794-kube-api-access-2gxpr\") pod \"odh-model-controller-7b966d7984-kvq8k\" (UID: \"95edd79f-f65d-4a62-bcbf-bef82c5ea794\") " pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.649384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.649345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:34.921914 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:34.921888    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-kvq8k"]
Sep 16 16:04:34.923931 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:04:34.923905    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95edd79f_f65d_4a62_bcbf_bef82c5ea794.slice/crio-618f022f95f876baaac2b28f6741f9cde31fd8a3d78f49153455f2dfaa818efd WatchSource:0}: Error finding container 618f022f95f876baaac2b28f6741f9cde31fd8a3d78f49153455f2dfaa818efd: Status 404 returned error can't find the container with id 618f022f95f876baaac2b28f6741f9cde31fd8a3d78f49153455f2dfaa818efd
Sep 16 16:04:35.178655 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:35.178555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-kvq8k" event={"ID":"95edd79f-f65d-4a62-bcbf-bef82c5ea794","Type":"ContainerStarted","Data":"618f022f95f876baaac2b28f6741f9cde31fd8a3d78f49153455f2dfaa818efd"}
Sep 16 16:04:39.204242 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:39.204202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-kvq8k" event={"ID":"95edd79f-f65d-4a62-bcbf-bef82c5ea794","Type":"ContainerStarted","Data":"66a56947f4f51b9d9ad2a0d40eec938a1759a8bfcfe091f4e1d6744a2243950a"}
Sep 16 16:04:39.204682 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:39.204293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:04:39.226670 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:39.226618    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-kvq8k" podStartSLOduration=1.028507518 podStartE2EDuration="5.226603001s" totalImagesPullingTime="4.198095483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:04:39.224140641 +0000 UTC m=+876.451598650" watchObservedRunningTime="2026-09-16 16:04:39.226603001 +0000 UTC m=+876.454061018"
Sep 16 16:04:50.213190 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:04:50.213126    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-kvq8k"
Sep 16 16:05:03.265384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:05:03.265298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:05:03.271009 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:05:03.270985    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:05:03.271505 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:05:03.271469    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:05:03.276865 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:05:03.276846    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:06:29.483991 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.483902    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:06:29.487530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.487508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.491782 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.491762    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-mfjsx\""
Sep 16 16:06:29.492129 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.491904    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Sep 16 16:06:29.492255 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.491873    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Sep 16 16:06:29.492255 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.491813    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Sep 16 16:06:29.492378 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.492097    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa-dockercfg-t286z\""
Sep 16 16:06:29.497198 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.497134    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:06:29.557788 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.557745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vfn\" (UniqueName: \"kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.557983 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.557801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.659053 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.659017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vfn\" (UniqueName: \"kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.659053 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.659059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.661629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.661600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.667519 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.667496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vfn\" (UniqueName: \"kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:29.801273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:29.801179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:30.085195 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:30.085151    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:06:30.087531 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:06:30.087495    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d70dec_762d_4e63_b1fa_b090c99b744a.slice/crio-93de1b84f44a94c7c9a80733e0d72caf50f843cbf9903e8d4cea74fc72d15675 WatchSource:0}: Error finding container 93de1b84f44a94c7c9a80733e0d72caf50f843cbf9903e8d4cea74fc72d15675: Status 404 returned error can't find the container with id 93de1b84f44a94c7c9a80733e0d72caf50f843cbf9903e8d4cea74fc72d15675
Sep 16 16:06:30.807946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:30.807903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" event={"ID":"a7d70dec-762d-4e63-b1fa-b090c99b744a","Type":"ContainerStarted","Data":"93de1b84f44a94c7c9a80733e0d72caf50f843cbf9903e8d4cea74fc72d15675"}
Sep 16 16:06:32.821453 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:32.821415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" event={"ID":"a7d70dec-762d-4e63-b1fa-b090c99b744a","Type":"ContainerStarted","Data":"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"}
Sep 16 16:06:32.821899 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:32.821527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:06:32.839501 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:32.839455    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podStartSLOduration=1.333596878 podStartE2EDuration="3.839441212s" totalImagesPullingTime="2.505844334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:06:32.837756071 +0000 UTC m=+990.065214077" watchObservedRunningTime="2026-09-16 16:06:32.839441212 +0000 UTC m=+990.066899218"
Sep 16 16:06:44.291256 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.291218    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:06:44.294605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.294584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.298364 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.298340    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Sep 16 16:06:44.298570 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.298545    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Sep 16 16:06:44.298699 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.298666    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-fd8sp\""
Sep 16 16:06:44.298772 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.298754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Sep 16 16:06:44.298828 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.298805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-k85rg\""
Sep 16 16:06:44.305046 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.305020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:06:44.391533 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.391494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbtwx\" (UniqueName: \"kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.391706 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.391553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.492738 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.492694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.492929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.492803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbtwx\" (UniqueName: \"kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.495417 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.495390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.500860 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.500836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbtwx\" (UniqueName: \"kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.608814 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.608733    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:44.884802 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:44.884768    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:06:44.886698 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:06:44.886666    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a529eb_8fa6_4c69_951b_d978680b37c5.slice/crio-8bf20a532365bc3fdd764b224aa142cc56823caed741c2f65a441cf5917b728c WatchSource:0}: Error finding container 8bf20a532365bc3fdd764b224aa142cc56823caed741c2f65a441cf5917b728c: Status 404 returned error can't find the container with id 8bf20a532365bc3fdd764b224aa142cc56823caed741c2f65a441cf5917b728c
Sep 16 16:06:45.896227 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:45.896189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" event={"ID":"28a529eb-8fa6-4c69-951b-d978680b37c5","Type":"ContainerStarted","Data":"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"}
Sep 16 16:06:45.896677 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:45.896243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" event={"ID":"28a529eb-8fa6-4c69-951b-d978680b37c5","Type":"ContainerStarted","Data":"8bf20a532365bc3fdd764b224aa142cc56823caed741c2f65a441cf5917b728c"}
Sep 16 16:06:45.896677 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:45.896390    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:06:45.914937 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:06:45.914886    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podStartSLOduration=1.914870007 podStartE2EDuration="1.914870007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:06:45.912006033 +0000 UTC m=+1003.139464041" watchObservedRunningTime="2026-09-16 16:06:45.914870007 +0000 UTC m=+1003.142328013"
Sep 16 16:07:03.829273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:03.829243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:07:16.903760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:16.903730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:07:23.303786 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:23.303746    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:07:26.904526 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:26.904487    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:07:36.903862 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:36.903821    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:07:38.336695 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:38.336651    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" containerID="cri-o://7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49" gracePeriod=45
Sep 16 16:07:46.904434 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:46.904396    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:07:46.904896 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:46.904512    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:07:56.904592 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:07:56.904495    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:08:06.903830 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:06.903789    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:08:08.597378 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.597313    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:08:08.716585 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.716549    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs\") pod \"28a529eb-8fa6-4c69-951b-d978680b37c5\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") "
Sep 16 16:08:08.716766 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.716641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbtwx\" (UniqueName: \"kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx\") pod \"28a529eb-8fa6-4c69-951b-d978680b37c5\" (UID: \"28a529eb-8fa6-4c69-951b-d978680b37c5\") "
Sep 16 16:08:08.718793 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.718764    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28a529eb-8fa6-4c69-951b-d978680b37c5" (UID: "28a529eb-8fa6-4c69-951b-d978680b37c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:08:08.718793 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.718785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx" (OuterVolumeSpecName: "kube-api-access-pbtwx") pod "28a529eb-8fa6-4c69-951b-d978680b37c5" (UID: "28a529eb-8fa6-4c69-951b-d978680b37c5"). InnerVolumeSpecName "kube-api-access-pbtwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:08:08.818049 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.818008    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a529eb-8fa6-4c69-951b-d978680b37c5-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:08:08.818049 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:08.818041    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbtwx\" (UniqueName: \"kubernetes.io/projected/28a529eb-8fa6-4c69-951b-d978680b37c5-kube-api-access-pbtwx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:08:09.359769 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.359740    2581 generic.go:356] "Generic (PLEG): container finished" podID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerID="7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49" exitCode=0
Sep 16 16:08:09.359904 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.359785    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"
Sep 16 16:08:09.359904 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.359822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" event={"ID":"28a529eb-8fa6-4c69-951b-d978680b37c5","Type":"ContainerDied","Data":"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"}
Sep 16 16:08:09.359904 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.359875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt" event={"ID":"28a529eb-8fa6-4c69-951b-d978680b37c5","Type":"ContainerDied","Data":"8bf20a532365bc3fdd764b224aa142cc56823caed741c2f65a441cf5917b728c"}
Sep 16 16:08:09.359904 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.359903    2581 scope.go:117] "RemoveContainer" containerID="7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"
Sep 16 16:08:09.369495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.369403    2581 scope.go:117] "RemoveContainer" containerID="7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"
Sep 16 16:08:09.369802 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:08:09.369740    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49\": container with ID starting with 7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49 not found: ID does not exist" containerID="7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"
Sep 16 16:08:09.369802 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.369774    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49"} err="failed to get container status \"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49\": rpc error: code = NotFound desc = could not find container \"7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49\": container with ID starting with 7d2c21bd378b2d7128daecbcf3309a89bf1b03afffcf1d846fe998424e16ad49 not found: ID does not exist"
Sep 16 16:08:09.378414 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.378389    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:08:09.381714 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:09.381693    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-7q4qt"]
Sep 16 16:08:11.310066 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:11.310030    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" path="/var/lib/kubelet/pods/28a529eb-8fa6-4c69-951b-d978680b37c5/volumes"
Sep 16 16:08:27.573171 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.573119    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:08:27.573595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.573574    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main"
Sep 16 16:08:27.573688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.573600    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main"
Sep 16 16:08:27.573776 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.573731    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a529eb-8fa6-4c69-951b-d978680b37c5" containerName="main"
Sep 16 16:08:27.578511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.578493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.582398 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.582369    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Sep 16 16:08:27.582584 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.582557    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Sep 16 16:08:27.582675 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.582623    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Sep 16 16:08:27.582675 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.582571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-m8xfs\""
Sep 16 16:08:27.589049 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.589026    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:08:27.675707 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.675895 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.675895 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.675895 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.675895 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.676040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.675900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95qlq\" (UniqueName: \"kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776634 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776634 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776879 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776879 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776879 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95qlq\" (UniqueName: \"kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.776879 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.776732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.777202 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.777141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.777316 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.777231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.777316 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.777235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.778958 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.778937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.779264 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.779246    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.785603 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.785579    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95qlq\" (UniqueName: \"kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq\") pod \"fc-concurrency-test-kserve-b5d44467c-pqb5l\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:27.890929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:27.890903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:28.164273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:28.164238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:08:28.165872 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:08:28.165839    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a4d29e_ee96_46c4_8569_9f657181bc78.slice/crio-322c128075b96a5acbe256e03ee68ce9ad2c8ef5ac10fa0cce4457fbd57d7933 WatchSource:0}: Error finding container 322c128075b96a5acbe256e03ee68ce9ad2c8ef5ac10fa0cce4457fbd57d7933: Status 404 returned error can't find the container with id 322c128075b96a5acbe256e03ee68ce9ad2c8ef5ac10fa0cce4457fbd57d7933
Sep 16 16:08:28.460831 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:28.460756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" event={"ID":"e6a4d29e-ee96-46c4-8569-9f657181bc78","Type":"ContainerStarted","Data":"322c128075b96a5acbe256e03ee68ce9ad2c8ef5ac10fa0cce4457fbd57d7933"}
Sep 16 16:08:30.477569 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:30.477527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" event={"ID":"e6a4d29e-ee96-46c4-8569-9f657181bc78","Type":"ContainerStarted","Data":"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"}
Sep 16 16:08:30.497051 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:30.497008    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" podStartSLOduration=1.958304677 podStartE2EDuration="3.496993606s" totalImagesPullingTime="1.538688929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:08:30.495404124 +0000 UTC m=+1107.722862133" watchObservedRunningTime="2026-09-16 16:08:30.496993606 +0000 UTC m=+1107.724451612"
Sep 16 16:08:30.891123 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:30.891092    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:37.891827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:37.891786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:37.904565 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:37.904538    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:38.529279 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:38.529244    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:08:48.384057 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:48.384015    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:08:53.828762 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:08:53.828719    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:09:03.404307 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:03.404246    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main" containerID="cri-o://b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468" gracePeriod=45
Sep 16 16:09:03.829301 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:03.829219    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:09:08.477557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.477514    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:09:08.490801 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:09:08.490753    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 16 16:09:08.490801 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:09:08.490801 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:09:08.490801 ip-10-0-137-111 kubenswrapper[2581]:  > containerID="ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c" cmd=["/bin/sleep","15"]
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.490826    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" message="" err=<
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:  >
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:09:08.490846    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]:  > pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main" containerID="cri-o://ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"
Sep 16 16:09:08.491151 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.490907    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main" containerID="cri-o://ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c" gracePeriod=60
Sep 16 16:09:08.518548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.518500    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": dial tcp 10.133.0.50:8000: connect: connection refused"
Sep 16 16:09:08.867521 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.867493    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:09:08.945394 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95qlq\" (UniqueName: \"kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945534    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home\") pod \"e6a4d29e-ee96-46c4-8569-9f657181bc78\" (UID: \"e6a4d29e-ee96-46c4-8569-9f657181bc78\") "
Sep 16 16:09:08.945867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:09:08.945867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:09:08.945867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945854    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:08.946031 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.945896    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home" (OuterVolumeSpecName: "home") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:09:08.947736 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.947704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm" (OuterVolumeSpecName: "dshm") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:09:08.947736 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.947709    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:09:08.947883 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:08.947817    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq" (OuterVolumeSpecName: "kube-api-access-95qlq") pod "e6a4d29e-ee96-46c4-8569-9f657181bc78" (UID: "e6a4d29e-ee96-46c4-8569-9f657181bc78"). InnerVolumeSpecName "kube-api-access-95qlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:09:09.046563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.046462    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:09.046563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.046494    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:09.046563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.046503    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a4d29e-ee96-46c4-8569-9f657181bc78-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:09.046563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.046512    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95qlq\" (UniqueName: \"kubernetes.io/projected/e6a4d29e-ee96-46c4-8569-9f657181bc78-kube-api-access-95qlq\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:09.046563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.046519    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a4d29e-ee96-46c4-8569-9f657181bc78-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:09.688093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.688054    2581 generic.go:356] "Generic (PLEG): container finished" podID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerID="ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c" exitCode=0
Sep 16 16:09:09.688484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.688101    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"
Sep 16 16:09:09.688484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.688145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" event={"ID":"e6a4d29e-ee96-46c4-8569-9f657181bc78","Type":"ContainerDied","Data":"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"}
Sep 16 16:09:09.688484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.688226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l" event={"ID":"e6a4d29e-ee96-46c4-8569-9f657181bc78","Type":"ContainerDied","Data":"322c128075b96a5acbe256e03ee68ce9ad2c8ef5ac10fa0cce4457fbd57d7933"}
Sep 16 16:09:09.688484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.688253    2581 scope.go:117] "RemoveContainer" containerID="ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"
Sep 16 16:09:09.697368 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.697348    2581 scope.go:117] "RemoveContainer" containerID="ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"
Sep 16 16:09:09.697630 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:09:09.697611    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c\": container with ID starting with ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c not found: ID does not exist" containerID="ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"
Sep 16 16:09:09.697703 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.697643    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c"} err="failed to get container status \"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c\": rpc error: code = NotFound desc = could not find container \"ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c\": container with ID starting with ea8bcc6d624cc74d796b81a9003f9b6d700cb2cf04f3719ee983f5eb2fbf083c not found: ID does not exist"
Sep 16 16:09:09.706041 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.706018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:09:09.710338 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:09.710317    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-b5d44467c-pqb5l"]
Sep 16 16:09:11.310375 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:11.310341    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" path="/var/lib/kubelet/pods/e6a4d29e-ee96-46c4-8569-9f657181bc78/volumes"
Sep 16 16:09:13.829357 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:13.829316    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:09:13.829832 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:13.829448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:09:23.828545 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:23.828504    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:09:33.666323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.666258    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:09:33.753699 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.753670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs\") pod \"a7d70dec-762d-4e63-b1fa-b090c99b744a\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") "
Sep 16 16:09:33.753892 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.753720    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69vfn\" (UniqueName: \"kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn\") pod \"a7d70dec-762d-4e63-b1fa-b090c99b744a\" (UID: \"a7d70dec-762d-4e63-b1fa-b090c99b744a\") "
Sep 16 16:09:33.755914 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.755886    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7d70dec-762d-4e63-b1fa-b090c99b744a" (UID: "a7d70dec-762d-4e63-b1fa-b090c99b744a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:09:33.756025 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.755966    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn" (OuterVolumeSpecName: "kube-api-access-69vfn") pod "a7d70dec-762d-4e63-b1fa-b090c99b744a" (UID: "a7d70dec-762d-4e63-b1fa-b090c99b744a"). InnerVolumeSpecName "kube-api-access-69vfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:09:33.815435 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.815397    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerID="b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468" exitCode=0
Sep 16 16:09:33.815620 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.815463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" event={"ID":"a7d70dec-762d-4e63-b1fa-b090c99b744a","Type":"ContainerDied","Data":"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"}
Sep 16 16:09:33.815620 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.815505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87" event={"ID":"a7d70dec-762d-4e63-b1fa-b090c99b744a","Type":"ContainerDied","Data":"93de1b84f44a94c7c9a80733e0d72caf50f843cbf9903e8d4cea74fc72d15675"}
Sep 16 16:09:33.815620 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.815510    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"
Sep 16 16:09:33.815620 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.815531    2581 scope.go:117] "RemoveContainer" containerID="b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"
Sep 16 16:09:33.824878 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.824858    2581 scope.go:117] "RemoveContainer" containerID="b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"
Sep 16 16:09:33.825121 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:09:33.825103    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468\": container with ID starting with b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468 not found: ID does not exist" containerID="b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"
Sep 16 16:09:33.825223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.825129    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468"} err="failed to get container status \"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468\": rpc error: code = NotFound desc = could not find container \"b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468\": container with ID starting with b35336dd2057426ca16a8e2fa655f5557583bc0a97cbe4254cd6fdd060979468 not found: ID does not exist"
Sep 16 16:09:33.837191 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.837144    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:09:33.841220 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.841196    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schec2z87"]
Sep 16 16:09:33.855025 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.855000    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d70dec-762d-4e63-b1fa-b090c99b744a-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:33.855025 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:33.855022    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69vfn\" (UniqueName: \"kubernetes.io/projected/a7d70dec-762d-4e63-b1fa-b090c99b744a-kube-api-access-69vfn\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:09:35.310172 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:35.310127    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" path="/var/lib/kubelet/pods/a7d70dec-762d-4e63-b1fa-b090c99b744a/volumes"
Sep 16 16:09:45.291963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.291930    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:09:45.292546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292515    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main"
Sep 16 16:09:45.292546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292538    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main"
Sep 16 16:09:45.292688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292554    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main"
Sep 16 16:09:45.292688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292563    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main"
Sep 16 16:09:45.292688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292682    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a4d29e-ee96-46c4-8569-9f657181bc78" containerName="main"
Sep 16 16:09:45.292851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.292694    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7d70dec-762d-4e63-b1fa-b090c99b744a" containerName="main"
Sep 16 16:09:45.297051 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.297031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.299703 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.299681    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\""
Sep 16 16:09:45.299802 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.299706    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\""
Sep 16 16:09:45.300993 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.300972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Sep 16 16:09:45.301082 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.300999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-epp-sa-dockercfg-pp8fh\""
Sep 16 16:09:45.301082 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.301055    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-7lwdm\""
Sep 16 16:09:45.311260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.311240    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:09:45.463122 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.463081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.463122 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.463119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vsx\" (UniqueName: \"kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.564199 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.564096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.564199 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.564136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vsx\" (UniqueName: \"kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.566618 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.566591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.576603 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.576571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vsx\" (UniqueName: \"kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx\") pod \"scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.608846 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.608806    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:45.895787 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:45.895757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:09:45.897805 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:09:45.897781    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0b9767_0309_4c1b_b1c1_93c95ec3faab.slice/crio-462c1d5b15c42b891b79a8d3acef4d69a0e936cd2c6ae6a8637f3cbdd50bfff9 WatchSource:0}: Error finding container 462c1d5b15c42b891b79a8d3acef4d69a0e936cd2c6ae6a8637f3cbdd50bfff9: Status 404 returned error can't find the container with id 462c1d5b15c42b891b79a8d3acef4d69a0e936cd2c6ae6a8637f3cbdd50bfff9
Sep 16 16:09:46.888988 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:46.888951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" event={"ID":"2f0b9767-0309-4c1b-b1c1-93c95ec3faab","Type":"ContainerStarted","Data":"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"}
Sep 16 16:09:46.889490 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:46.889011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:09:46.889490 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:46.889027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" event={"ID":"2f0b9767-0309-4c1b-b1c1-93c95ec3faab","Type":"ContainerStarted","Data":"462c1d5b15c42b891b79a8d3acef4d69a0e936cd2c6ae6a8637f3cbdd50bfff9"}
Sep 16 16:09:46.907545 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:09:46.907495    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podStartSLOduration=1.907478751 podStartE2EDuration="1.907478751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:09:46.905545829 +0000 UTC m=+1184.133003835" watchObservedRunningTime="2026-09-16 16:09:46.907478751 +0000 UTC m=+1184.134936757"
Sep 16 16:10:03.311728 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:03.311701    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:10:03.313688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:03.313667    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:10:03.317468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:03.317452    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:10:03.319243 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:03.319225    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:10:17.896166 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:17.896118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:10:55.434147 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:55.434055    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:10:57.896613 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:10:57.896574    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:11:07.896990 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:07.896945    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:11:10.456568 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:10.456501    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" containerID="cri-o://b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0" gracePeriod=45
Sep 16 16:11:17.896107 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:17.896059    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:11:17.896514 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:17.896187    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:11:27.896979 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:27.896930    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:11:37.896984 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:37.896947    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:11:40.710486 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.710450    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:11:40.757807 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.757710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs\") pod \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") "
Sep 16 16:11:40.757807 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.757764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4vsx\" (UniqueName: \"kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx\") pod \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\" (UID: \"2f0b9767-0309-4c1b-b1c1-93c95ec3faab\") "
Sep 16 16:11:40.759989 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.759964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f0b9767-0309-4c1b-b1c1-93c95ec3faab" (UID: "2f0b9767-0309-4c1b-b1c1-93c95ec3faab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:11:40.760117 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.760008    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx" (OuterVolumeSpecName: "kube-api-access-g4vsx") pod "2f0b9767-0309-4c1b-b1c1-93c95ec3faab" (UID: "2f0b9767-0309-4c1b-b1c1-93c95ec3faab"). InnerVolumeSpecName "kube-api-access-g4vsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:11:40.859033 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.858991    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:11:40.859033 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:40.859028    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4vsx\" (UniqueName: \"kubernetes.io/projected/2f0b9767-0309-4c1b-b1c1-93c95ec3faab-kube-api-access-g4vsx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:11:41.505044 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.505009    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerID="b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0" exitCode=0
Sep 16 16:11:41.505247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.505057    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"
Sep 16 16:11:41.505247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.505050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" event={"ID":"2f0b9767-0309-4c1b-b1c1-93c95ec3faab","Type":"ContainerDied","Data":"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"}
Sep 16 16:11:41.505247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.505130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29" event={"ID":"2f0b9767-0309-4c1b-b1c1-93c95ec3faab","Type":"ContainerDied","Data":"462c1d5b15c42b891b79a8d3acef4d69a0e936cd2c6ae6a8637f3cbdd50bfff9"}
Sep 16 16:11:41.505247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.505180    2581 scope.go:117] "RemoveContainer" containerID="b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"
Sep 16 16:11:41.514082 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.514063    2581 scope.go:117] "RemoveContainer" containerID="b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"
Sep 16 16:11:41.514354 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:11:41.514332    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0\": container with ID starting with b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0 not found: ID does not exist" containerID="b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"
Sep 16 16:11:41.514408 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.514360    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0"} err="failed to get container status \"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0\": rpc error: code = NotFound desc = could not find container \"b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0\": container with ID starting with b56ddc77ffb586a91da742fa063d89b9090af058d8dfcf940d20bcca70882dd0 not found: ID does not exist"
Sep 16 16:11:41.521860 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.521837    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:11:41.525997 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:41.525976    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-695cb2mc29"]
Sep 16 16:11:43.310705 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:11:43.310671    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" path="/var/lib/kubelet/pods/2f0b9767-0309-4c1b-b1c1-93c95ec3faab/volumes"
Sep 16 16:12:00.034140 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.034105    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:12:00.034725 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.034469    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main"
Sep 16 16:12:00.034725 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.034481    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main"
Sep 16 16:12:00.034725 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.034558    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0b9767-0309-4c1b-b1c1-93c95ec3faab" containerName="main"
Sep 16 16:12:00.039230 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.039204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.042266 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.042241    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-xm62p\""
Sep 16 16:12:00.043566 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.043545    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Sep 16 16:12:00.043668 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.043565    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-xvt2d\""
Sep 16 16:12:00.043668 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.043600    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Sep 16 16:12:00.043668 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.043565    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Sep 16 16:12:00.053204 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.053177    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:12:00.130402 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.130361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s5p8\" (UniqueName: \"kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.130586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.130449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.231139 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.231098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.231341 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.231192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s5p8\" (UniqueName: \"kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.233830 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.233801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.240714 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.240691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s5p8\" (UniqueName: \"kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.349929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.349837    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:00.654221 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:12:00.654151    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f0dca1_15b3_43b5_9bec_0a0f6e9cfb26.slice/crio-31b953f88be66b7055e59eade4d463769ddf1365e37e5344fc2fb0cc83bd9d9c WatchSource:0}: Error finding container 31b953f88be66b7055e59eade4d463769ddf1365e37e5344fc2fb0cc83bd9d9c: Status 404 returned error can't find the container with id 31b953f88be66b7055e59eade4d463769ddf1365e37e5344fc2fb0cc83bd9d9c
Sep 16 16:12:00.654872 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:00.654843    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:12:01.616824 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:01.616780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" event={"ID":"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26","Type":"ContainerStarted","Data":"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"}
Sep 16 16:12:01.617317 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:01.616839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" event={"ID":"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26","Type":"ContainerStarted","Data":"31b953f88be66b7055e59eade4d463769ddf1365e37e5344fc2fb0cc83bd9d9c"}
Sep 16 16:12:01.617317 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:01.616860    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:01.635800 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:01.635746    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podStartSLOduration=1.635731191 podStartE2EDuration="1.635731191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:12:01.633971703 +0000 UTC m=+1318.861429720" watchObservedRunningTime="2026-09-16 16:12:01.635731191 +0000 UTC m=+1318.863189197"
Sep 16 16:12:32.624483 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:32.624400    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:12:34.150903 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:34.150867    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:12:42.624271 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:42.624230    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:12:49.170128 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:49.170061    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main" containerID="cri-o://72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb" gracePeriod=45
Sep 16 16:12:52.623762 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:12:52.623723    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:13:02.624195 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:02.624133    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:13:02.624672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:02.624279    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:13:12.624486 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:12.624448    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:13:19.412437 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.412411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:13:19.512201 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.512110    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s5p8\" (UniqueName: \"kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8\") pod \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") "
Sep 16 16:13:19.512201 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.512141    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs\") pod \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\" (UID: \"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26\") "
Sep 16 16:13:19.514300 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.514273    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8" (OuterVolumeSpecName: "kube-api-access-2s5p8") pod "f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" (UID: "f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26"). InnerVolumeSpecName "kube-api-access-2s5p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:13:19.514443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.514378    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" (UID: "f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:13:19.612985 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.612930    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s5p8\" (UniqueName: \"kubernetes.io/projected/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-kube-api-access-2s5p8\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:13:19.612985 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:19.612978    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:13:20.026578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.026542    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerID="72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb" exitCode=0
Sep 16 16:13:20.026749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.026590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"
Sep 16 16:13:20.026749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.026629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" event={"ID":"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26","Type":"ContainerDied","Data":"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"}
Sep 16 16:13:20.026749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.026684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw" event={"ID":"f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26","Type":"ContainerDied","Data":"31b953f88be66b7055e59eade4d463769ddf1365e37e5344fc2fb0cc83bd9d9c"}
Sep 16 16:13:20.026749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.026715    2581 scope.go:117] "RemoveContainer" containerID="72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"
Sep 16 16:13:20.035854 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.035832    2581 scope.go:117] "RemoveContainer" containerID="72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"
Sep 16 16:13:20.036113 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:13:20.036096    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb\": container with ID starting with 72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb not found: ID does not exist" containerID="72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"
Sep 16 16:13:20.036227 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.036120    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb"} err="failed to get container status \"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb\": rpc error: code = NotFound desc = could not find container \"72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb\": container with ID starting with 72fdc9675d85c2e6cb13209b0f3d2923f2aabb35c8c37afade54c1b540cbc0eb not found: ID does not exist"
Sep 16 16:13:20.048730 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.048709    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:13:20.053399 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:20.053377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scherskcw"]
Sep 16 16:13:21.318643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:21.318609    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" path="/var/lib/kubelet/pods/f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26/volumes"
Sep 16 16:13:39.645937 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.645886    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:13:39.646518 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.646494    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main"
Sep 16 16:13:39.646518 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.646518    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main"
Sep 16 16:13:39.646679 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.646621    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f0dca1-15b3-43b5-9bec-0a0f6e9cfb26" containerName="main"
Sep 16 16:13:39.652371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.652352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.656887 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.656858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-4bdwq\""
Sep 16 16:13:39.656887 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.656884    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Sep 16 16:13:39.657062 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.656903    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-xjqxq\""
Sep 16 16:13:39.657062 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.656858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Sep 16 16:13:39.657208 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.657066    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Sep 16 16:13:39.660567 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.660197    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:13:39.687416 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.687375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jv9t\" (UniqueName: \"kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.687569 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.687492    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.788493 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.788453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jv9t\" (UniqueName: \"kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.788683 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.788542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.791078 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.791055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.796988 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.796963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jv9t\" (UniqueName: \"kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:39.964202 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:39.964096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:40.248008 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:40.247961    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:13:40.248987 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:13:40.248955    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode881f411_8d6f_4036_a68a_f4fc09395340.slice/crio-e1def3ccb29aff61440c460eb53e8526d0f05c22e173c6aecda15b06f4795f74 WatchSource:0}: Error finding container e1def3ccb29aff61440c460eb53e8526d0f05c22e173c6aecda15b06f4795f74: Status 404 returned error can't find the container with id e1def3ccb29aff61440c460eb53e8526d0f05c22e173c6aecda15b06f4795f74
Sep 16 16:13:41.140296 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:41.140259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" event={"ID":"e881f411-8d6f-4036-a68a-f4fc09395340","Type":"ContainerStarted","Data":"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"}
Sep 16 16:13:41.140755 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:41.140310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" event={"ID":"e881f411-8d6f-4036-a68a-f4fc09395340","Type":"ContainerStarted","Data":"e1def3ccb29aff61440c460eb53e8526d0f05c22e173c6aecda15b06f4795f74"}
Sep 16 16:13:41.140755 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:41.140351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:13:41.159470 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:41.159422    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podStartSLOduration=2.1594077130000002 podStartE2EDuration="2.159407713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:13:41.157632158 +0000 UTC m=+1418.385090165" watchObservedRunningTime="2026-09-16 16:13:41.159407713 +0000 UTC m=+1418.386865718"
Sep 16 16:13:55.178079 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.178047    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:13:55.180874 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.180853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.183826 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.183803    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Sep 16 16:13:55.185428 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.185404    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-rhf8d\""
Sep 16 16:13:55.185536 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.185427    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Sep 16 16:13:55.185536 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.185427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Sep 16 16:13:55.185536 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.185510    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-g2g48\""
Sep 16 16:13:55.190528 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.190505    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:13:55.320397 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.320366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjslg\" (UniqueName: \"kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.320571 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.320408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.421276 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.421184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjslg\" (UniqueName: \"kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.421276 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.421259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.423928 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.423901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.429575 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.429524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjslg\" (UniqueName: \"kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg\") pod \"auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.491671 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.491634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:55.777852 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:55.777824    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:13:55.780112 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:13:55.780075    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b77ba44_4459_41d8_9a68_d93df42e05a7.slice/crio-2f25557656a0ca9dc740d1c18903ad29af9285bcd8ad6d7fecbaf751ce03dfa2 WatchSource:0}: Error finding container 2f25557656a0ca9dc740d1c18903ad29af9285bcd8ad6d7fecbaf751ce03dfa2: Status 404 returned error can't find the container with id 2f25557656a0ca9dc740d1c18903ad29af9285bcd8ad6d7fecbaf751ce03dfa2
Sep 16 16:13:56.221742 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:56.221704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" event={"ID":"7b77ba44-4459-41d8-9a68-d93df42e05a7","Type":"ContainerStarted","Data":"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"}
Sep 16 16:13:56.222103 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:56.221761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" event={"ID":"7b77ba44-4459-41d8-9a68-d93df42e05a7","Type":"ContainerStarted","Data":"2f25557656a0ca9dc740d1c18903ad29af9285bcd8ad6d7fecbaf751ce03dfa2"}
Sep 16 16:13:56.222103 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:56.221802    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:13:56.243385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:13:56.243333    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podStartSLOduration=1.243315829 podStartE2EDuration="1.243315829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:13:56.242103821 +0000 UTC m=+1433.469561829" watchObservedRunningTime="2026-09-16 16:13:56.243315829 +0000 UTC m=+1433.470773834"
Sep 16 16:14:12.147539 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:12.147504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:14:13.914551 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:13.914518    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:14:22.147715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:22.147670    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:14:27.229314 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:27.229281    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:14:28.937273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:28.937231    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main" containerID="cri-o://8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747" gracePeriod=45
Sep 16 16:14:32.147101 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:32.147053    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:14:42.147601 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:42.147556    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:14:42.147978 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:42.147663    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:14:52.147281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:52.147241    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:14:59.187751 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.187728    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:14:59.369270 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.369180    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jv9t\" (UniqueName: \"kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t\") pod \"e881f411-8d6f-4036-a68a-f4fc09395340\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") "
Sep 16 16:14:59.369270 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.369236    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs\") pod \"e881f411-8d6f-4036-a68a-f4fc09395340\" (UID: \"e881f411-8d6f-4036-a68a-f4fc09395340\") "
Sep 16 16:14:59.371495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.371460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e881f411-8d6f-4036-a68a-f4fc09395340" (UID: "e881f411-8d6f-4036-a68a-f4fc09395340"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:14:59.371495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.371461    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t" (OuterVolumeSpecName: "kube-api-access-4jv9t") pod "e881f411-8d6f-4036-a68a-f4fc09395340" (UID: "e881f411-8d6f-4036-a68a-f4fc09395340"). InnerVolumeSpecName "kube-api-access-4jv9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:14:59.470756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.470716    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jv9t\" (UniqueName: \"kubernetes.io/projected/e881f411-8d6f-4036-a68a-f4fc09395340-kube-api-access-4jv9t\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:14:59.470756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.470747    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e881f411-8d6f-4036-a68a-f4fc09395340-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:14:59.557047 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.557015    2581 generic.go:356] "Generic (PLEG): container finished" podID="e881f411-8d6f-4036-a68a-f4fc09395340" containerID="8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747" exitCode=0
Sep 16 16:14:59.557224 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.557063    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"
Sep 16 16:14:59.557224 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.557105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" event={"ID":"e881f411-8d6f-4036-a68a-f4fc09395340","Type":"ContainerDied","Data":"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"}
Sep 16 16:14:59.557224 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.557169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5" event={"ID":"e881f411-8d6f-4036-a68a-f4fc09395340","Type":"ContainerDied","Data":"e1def3ccb29aff61440c460eb53e8526d0f05c22e173c6aecda15b06f4795f74"}
Sep 16 16:14:59.557224 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.557197    2581 scope.go:117] "RemoveContainer" containerID="8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"
Sep 16 16:14:59.566781 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.566761    2581 scope.go:117] "RemoveContainer" containerID="8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"
Sep 16 16:14:59.567077 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:14:59.567058    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747\": container with ID starting with 8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747 not found: ID does not exist" containerID="8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"
Sep 16 16:14:59.567123 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.567086    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747"} err="failed to get container status \"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747\": rpc error: code = NotFound desc = could not find container \"8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747\": container with ID starting with 8409d509ec3fb22fb49dc989d4d4834203d772b6fd8ad95950f7784de41a5747 not found: ID does not exist"
Sep 16 16:14:59.579871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.579847    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:14:59.583567 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:14:59.583548    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-566c9jnnh5"]
Sep 16 16:15:01.310371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:01.310330    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" path="/var/lib/kubelet/pods/e881f411-8d6f-4036-a68a-f4fc09395340/volumes"
Sep 16 16:15:03.351618 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:03.351588    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:15:03.352022 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:03.351963    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:15:03.357382 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:03.357360    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:15:03.357579 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:03.357558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:15:08.256081 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.256048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:15:08.256454 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.256437    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main"
Sep 16 16:15:08.256454 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.256450    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main"
Sep 16 16:15:08.256524 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.256504    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e881f411-8d6f-4036-a68a-f4fc09395340" containerName="main"
Sep 16 16:15:08.260526 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.260506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.264544 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.264522    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Sep 16 16:15:08.264739 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.264719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Sep 16 16:15:08.265281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.264651    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-9xtwm\""
Sep 16 16:15:08.265281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.264684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-tdtgz\""
Sep 16 16:15:08.265281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.264577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Sep 16 16:15:08.267873 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.267814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:15:08.442607 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.442575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.442789 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.442616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945nx\" (UniqueName: \"kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.543525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.543428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.543525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.543481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-945nx\" (UniqueName: \"kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.546071 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.546043    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.551956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.551924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-945nx\" (UniqueName: \"kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.572373 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.572346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:08.848233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:08.848133    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:15:08.851407 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:15:08.851365    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913acdb0_14e6_42e0_9bff_7b5438ad7ea4.slice/crio-5e5e21060a5469bdf006f604bc8b385e03daa017f65de9d2e2f296dfbae2dc1a WatchSource:0}: Error finding container 5e5e21060a5469bdf006f604bc8b385e03daa017f65de9d2e2f296dfbae2dc1a: Status 404 returned error can't find the container with id 5e5e21060a5469bdf006f604bc8b385e03daa017f65de9d2e2f296dfbae2dc1a
Sep 16 16:15:09.612195 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:09.612136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" event={"ID":"913acdb0-14e6-42e0-9bff-7b5438ad7ea4","Type":"ContainerStarted","Data":"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"}
Sep 16 16:15:09.612602 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:09.612210    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" event={"ID":"913acdb0-14e6-42e0-9bff-7b5438ad7ea4","Type":"ContainerStarted","Data":"5e5e21060a5469bdf006f604bc8b385e03daa017f65de9d2e2f296dfbae2dc1a"}
Sep 16 16:15:09.612602 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:09.612322    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:15:09.632920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:09.632862    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podStartSLOduration=1.632847034 podStartE2EDuration="1.632847034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:15:09.630458632 +0000 UTC m=+1506.857916640" watchObservedRunningTime="2026-09-16 16:15:09.632847034 +0000 UTC m=+1506.860305040"
Sep 16 16:15:40.620496 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:40.620411    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:15:41.460746 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:41.460710    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:15:41.462555 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:41.462520    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:15:51.463076 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:51.463036    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:15:56.490515 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:56.490463    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" containerID="cri-o://a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5" gracePeriod=45
Sep 16 16:15:56.521050 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:56.521010    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:15:57.229546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:15:57.229507    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:01.462758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:01.462720    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:07.229002 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:07.228964    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:11.462755 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:11.462713    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:11.541541 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:11.541498    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" containerID="cri-o://3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a" gracePeriod=45
Sep 16 16:16:17.229450 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:17.229410    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:17.229929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:17.229528    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:16:21.462323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:21.462281    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:26.752595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.752529    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:16:26.819756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.819718    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-945nx\" (UniqueName: \"kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx\") pod \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") "
Sep 16 16:16:26.819928 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.819770    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs\") pod \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\" (UID: \"913acdb0-14e6-42e0-9bff-7b5438ad7ea4\") "
Sep 16 16:16:26.822051 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.822019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "913acdb0-14e6-42e0-9bff-7b5438ad7ea4" (UID: "913acdb0-14e6-42e0-9bff-7b5438ad7ea4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:16:26.822183 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.822082    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx" (OuterVolumeSpecName: "kube-api-access-945nx") pod "913acdb0-14e6-42e0-9bff-7b5438ad7ea4" (UID: "913acdb0-14e6-42e0-9bff-7b5438ad7ea4"). InnerVolumeSpecName "kube-api-access-945nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:16:26.920672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.920635    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-945nx\" (UniqueName: \"kubernetes.io/projected/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-kube-api-access-945nx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:16:26.920672 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:26.920666    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/913acdb0-14e6-42e0-9bff-7b5438ad7ea4-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:16:27.048818 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.048734    2581 generic.go:356] "Generic (PLEG): container finished" podID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerID="a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5" exitCode=0
Sep 16 16:16:27.048818 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.048788    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"
Sep 16 16:16:27.048818 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.048801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" event={"ID":"913acdb0-14e6-42e0-9bff-7b5438ad7ea4","Type":"ContainerDied","Data":"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"}
Sep 16 16:16:27.049054 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.048891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p" event={"ID":"913acdb0-14e6-42e0-9bff-7b5438ad7ea4","Type":"ContainerDied","Data":"5e5e21060a5469bdf006f604bc8b385e03daa017f65de9d2e2f296dfbae2dc1a"}
Sep 16 16:16:27.049054 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.048915    2581 scope.go:117] "RemoveContainer" containerID="a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"
Sep 16 16:16:27.058600 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.058582    2581 scope.go:117] "RemoveContainer" containerID="a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"
Sep 16 16:16:27.058881 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:16:27.058863    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5\": container with ID starting with a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5 not found: ID does not exist" containerID="a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"
Sep 16 16:16:27.058921 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.058892    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5"} err="failed to get container status \"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5\": rpc error: code = NotFound desc = could not find container \"a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5\": container with ID starting with a2f20a9fe2a17fdcee80180f101f8644639ec577e4f591cfe626d5f930397fa5 not found: ID does not exist"
Sep 16 16:16:27.071959 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.071933    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:16:27.073948 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.073929    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-557bbc58dt2p"]
Sep 16 16:16:27.229468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.229430    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:27.310513 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:27.310428    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" path="/var/lib/kubelet/pods/913acdb0-14e6-42e0-9bff-7b5438ad7ea4/volumes"
Sep 16 16:16:37.229578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:37.229539    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:16:41.795055 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:41.795021    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:16:41.952562 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:41.952522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs\") pod \"7b77ba44-4459-41d8-9a68-d93df42e05a7\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") "
Sep 16 16:16:41.952743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:41.952591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjslg\" (UniqueName: \"kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg\") pod \"7b77ba44-4459-41d8-9a68-d93df42e05a7\" (UID: \"7b77ba44-4459-41d8-9a68-d93df42e05a7\") "
Sep 16 16:16:41.954819 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:41.954786    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg" (OuterVolumeSpecName: "kube-api-access-cjslg") pod "7b77ba44-4459-41d8-9a68-d93df42e05a7" (UID: "7b77ba44-4459-41d8-9a68-d93df42e05a7"). InnerVolumeSpecName "kube-api-access-cjslg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:16:41.954957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:41.954853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b77ba44-4459-41d8-9a68-d93df42e05a7" (UID: "7b77ba44-4459-41d8-9a68-d93df42e05a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:16:42.053708 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.053673    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b77ba44-4459-41d8-9a68-d93df42e05a7-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:16:42.053708 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.053703    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjslg\" (UniqueName: \"kubernetes.io/projected/7b77ba44-4459-41d8-9a68-d93df42e05a7-kube-api-access-cjslg\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:16:42.135789 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.135749    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerID="3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a" exitCode=0
Sep 16 16:16:42.135956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.135802    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"
Sep 16 16:16:42.135956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.135827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" event={"ID":"7b77ba44-4459-41d8-9a68-d93df42e05a7","Type":"ContainerDied","Data":"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"}
Sep 16 16:16:42.135956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.135884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd" event={"ID":"7b77ba44-4459-41d8-9a68-d93df42e05a7","Type":"ContainerDied","Data":"2f25557656a0ca9dc740d1c18903ad29af9285bcd8ad6d7fecbaf751ce03dfa2"}
Sep 16 16:16:42.135956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.135912    2581 scope.go:117] "RemoveContainer" containerID="3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"
Sep 16 16:16:42.145525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.145503    2581 scope.go:117] "RemoveContainer" containerID="3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"
Sep 16 16:16:42.145800 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:16:42.145781    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a\": container with ID starting with 3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a not found: ID does not exist" containerID="3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"
Sep 16 16:16:42.145886 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.145807    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a"} err="failed to get container status \"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a\": rpc error: code = NotFound desc = could not find container \"3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a\": container with ID starting with 3943736c35070add5d1d8f3144994a7d7e2957172405ea6c8f05a10c91ff497a not found: ID does not exist"
Sep 16 16:16:42.159007 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.158979    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:16:42.162257 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:42.162234    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6dc5d5cf9c4pvsd"]
Sep 16 16:16:43.023808 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.023776    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:16:43.024177 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024138    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main"
Sep 16 16:16:43.024177 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024150    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main"
Sep 16 16:16:43.024260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024180    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main"
Sep 16 16:16:43.024260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024187    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main"
Sep 16 16:16:43.024260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024255    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" containerName="main"
Sep 16 16:16:43.024356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.024263    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="913acdb0-14e6-42e0-9bff-7b5438ad7ea4" containerName="main"
Sep 16 16:16:43.028281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.028248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.032492 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.032463    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Sep 16 16:16:43.032492 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.032480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-epp-sa-dockercfg-cxh8n\""
Sep 16 16:16:43.032686 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.032481    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-hd65n\""
Sep 16 16:16:43.032686 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.032489    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Sep 16 16:16:43.032686 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.032579    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Sep 16 16:16:43.037801 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.037780    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:16:43.163233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.163194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.163443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.163255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.163443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.163306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.163443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.163366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6gh\" (UniqueName: \"kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.263975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6gh\" (UniqueName: \"kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264220 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.264042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264220 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.264078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264220 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.264096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264561 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.264540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.264619 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.264571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.266511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.266489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.271963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.271940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6gh\" (UniqueName: \"kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh\") pod \"scheduler-custom-template-test-kserve-router-scheduler-ffck74b8\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.310588 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.310506    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b77ba44-4459-41d8-9a68-d93df42e05a7" path="/var/lib/kubelet/pods/7b77ba44-4459-41d8-9a68-d93df42e05a7/volumes"
Sep 16 16:16:43.340271 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.340241    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:16:43.614417 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:43.614390    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:16:43.616320 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:16:43.616280    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb34adb_1b4a_4e3e_a5f6_ff00cdef08b9.slice/crio-53213c1cfe2ff1b117245fe1ec7299058da9745f80ef22a7a1d79f0ee9d46466 WatchSource:0}: Error finding container 53213c1cfe2ff1b117245fe1ec7299058da9745f80ef22a7a1d79f0ee9d46466: Status 404 returned error can't find the container with id 53213c1cfe2ff1b117245fe1ec7299058da9745f80ef22a7a1d79f0ee9d46466
Sep 16 16:16:44.150743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:44.150705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" event={"ID":"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9","Type":"ContainerStarted","Data":"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"}
Sep 16 16:16:44.151153 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:44.150761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" event={"ID":"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9","Type":"ContainerStarted","Data":"53213c1cfe2ff1b117245fe1ec7299058da9745f80ef22a7a1d79f0ee9d46466"}
Sep 16 16:16:44.151153 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:16:44.150794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:17:15.158478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:15.158445    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:17:15.177328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:15.177275    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podStartSLOduration=32.177259602 podStartE2EDuration="32.177259602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:16:44.168499222 +0000 UTC m=+1601.395957232" watchObservedRunningTime="2026-09-16 16:17:15.177259602 +0000 UTC m=+1632.404717618"
Sep 16 16:17:33.117314 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:33.117279    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:17:35.158115 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:35.158073    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:17:45.158206 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:45.158140    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:17:48.137743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:48.137671    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" containerID="cri-o://a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559" gracePeriod=45
Sep 16 16:17:55.158079 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:55.158037    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:17:55.158483 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:17:55.158228    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:18:05.158283 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:05.158242    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:18:15.158060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:15.158017    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:18:18.389908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.389881    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:18:18.488674 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.488582    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz6gh\" (UniqueName: \"kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh\") pod \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") "
Sep 16 16:18:18.488674 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.488632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs\") pod \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") "
Sep 16 16:18:18.488869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.488726    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache\") pod \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") "
Sep 16 16:18:18.488869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.488753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache\") pod \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\" (UID: \"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9\") "
Sep 16 16:18:18.489060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.489025    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" (UID: "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:18:18.489060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.489051    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" (UID: "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:18:18.490912 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.490884    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" (UID: "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:18:18.490997 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.490967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh" (OuterVolumeSpecName: "kube-api-access-nz6gh") pod "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" (UID: "fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9"). InnerVolumeSpecName "kube-api-access-nz6gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:18:18.589587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.589540    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz6gh\" (UniqueName: \"kubernetes.io/projected/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-kube-api-access-nz6gh\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:18:18.589587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.589583    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:18:18.589587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.589593    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-tokenizer-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:18:18.589587 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.589603    2581 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9-cachi2-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:18:18.643894 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.643861    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerID="a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559" exitCode=0
Sep 16 16:18:18.644086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.643902    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"
Sep 16 16:18:18.644086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.643949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" event={"ID":"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9","Type":"ContainerDied","Data":"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"}
Sep 16 16:18:18.644086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.644004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8" event={"ID":"fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9","Type":"ContainerDied","Data":"53213c1cfe2ff1b117245fe1ec7299058da9745f80ef22a7a1d79f0ee9d46466"}
Sep 16 16:18:18.644086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.644036    2581 scope.go:117] "RemoveContainer" containerID="a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"
Sep 16 16:18:18.653080 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.653062    2581 scope.go:117] "RemoveContainer" containerID="a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"
Sep 16 16:18:18.653389 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:18:18.653371    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559\": container with ID starting with a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559 not found: ID does not exist" containerID="a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"
Sep 16 16:18:18.653440 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.653399    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559"} err="failed to get container status \"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559\": rpc error: code = NotFound desc = could not find container \"a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559\": container with ID starting with a22810494f78a7b3e3a5ddacbed0a9fb3987f5c6f1f7f833903f785a25d6a559 not found: ID does not exist"
Sep 16 16:18:18.667896 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.667871    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:18:18.671440 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:18.671416    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-ffck74b8"]
Sep 16 16:18:19.316643 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:19.316610    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" path="/var/lib/kubelet/pods/fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9/volumes"
Sep 16 16:18:25.769385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.769302    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:18:25.769872 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.769850    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main"
Sep 16 16:18:25.769932 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.769878    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main"
Sep 16 16:18:25.770009 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.769997    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb34adb-1b4a-4e3e-a5f6-ff00cdef08b9" containerName="main"
Sep 16 16:18:25.774561 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.774538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.781605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.781584    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Sep 16 16:18:25.781856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.781836    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-z7hqw\""
Sep 16 16:18:25.785546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.785526    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Sep 16 16:18:25.787357 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.787324    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Sep 16 16:18:25.787893 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.787872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:18:25.853745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.853946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.853946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.853946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmcsp\" (UniqueName: \"kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.853946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.853946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.853887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.954794 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.954748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.954794 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.954798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.954831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.954935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmcsp\" (UniqueName: \"kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.954998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.955026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955363 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.955340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955427 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.955340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.955427 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.955376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.957091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.957067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.957430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.957412    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:25.964119 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:25.964097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmcsp\" (UniqueName: \"kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:26.070298 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.070209    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:18:26.073895 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.073875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.076678 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.076655    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-x4rpx\""
Sep 16 16:18:26.083956 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.083926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:18:26.084993 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.084977    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:26.156869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.156833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.157019 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.156912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbr8\" (UniqueName: \"kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.257680 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.257639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.257896 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.257699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbr8\" (UniqueName: \"kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.260404 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.260378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.266667 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.266626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbr8\" (UniqueName: \"kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.360063 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.360034    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:18:26.361962 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:18:26.361938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92d1409_3c2b_4353_a7df_3d98da39c125.slice/crio-96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e WatchSource:0}: Error finding container 96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e: Status 404 returned error can't find the container with id 96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e
Sep 16 16:18:26.385486 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.385462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:26.659693 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.659663    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:18:26.661770 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:18:26.661726    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae325778_cade_43d0_921e_564a2a4b7e74.slice/crio-e517cdcc88b8b0328abfd2e44d0c719d2dc1fed0837457ac7b6270da578d04c6 WatchSource:0}: Error finding container e517cdcc88b8b0328abfd2e44d0c719d2dc1fed0837457ac7b6270da578d04c6: Status 404 returned error can't find the container with id e517cdcc88b8b0328abfd2e44d0c719d2dc1fed0837457ac7b6270da578d04c6
Sep 16 16:18:26.691694 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.691662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" event={"ID":"ae325778-cade-43d0-921e-564a2a4b7e74","Type":"ContainerStarted","Data":"e517cdcc88b8b0328abfd2e44d0c719d2dc1fed0837457ac7b6270da578d04c6"}
Sep 16 16:18:26.693246 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.693222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" event={"ID":"d92d1409-3c2b-4353-a7df-3d98da39c125","Type":"ContainerStarted","Data":"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"}
Sep 16 16:18:26.693365 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.693270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" event={"ID":"d92d1409-3c2b-4353-a7df-3d98da39c125","Type":"ContainerStarted","Data":"96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e"}
Sep 16 16:18:26.711172 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:26.711111    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" podStartSLOduration=1.711096511 podStartE2EDuration="1.711096511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:18:26.710730233 +0000 UTC m=+1703.938188243" watchObservedRunningTime="2026-09-16 16:18:26.711096511 +0000 UTC m=+1703.938554520"
Sep 16 16:18:27.085921 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:27.085824    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:27.701859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:27.701823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" event={"ID":"ae325778-cade-43d0-921e-564a2a4b7e74","Type":"ContainerStarted","Data":"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"}
Sep 16 16:18:27.702047 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:27.701999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:18:27.721949 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:27.721891    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podStartSLOduration=1.721874794 podStartE2EDuration="1.721874794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:18:27.721264346 +0000 UTC m=+1704.948722356" watchObservedRunningTime="2026-09-16 16:18:27.721874794 +0000 UTC m=+1704.949332800"
Sep 16 16:18:36.085738 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:36.085695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:36.098754 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:36.098720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:36.764335 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:36.764298    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:18:58.710183 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:18:58.710130    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:20:03.391019 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:03.390988    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:20:03.391542 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:03.391481    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:20:03.403530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:03.403504    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:20:03.403934 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:03.403913    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:20:06.164307 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.164266    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:20:06.172090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.172064    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:20:06.175858 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:20:06.175818    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 16 16:20:06.175858 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:20:06.175858 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:20:06.175858 ip-10-0-137-111 kubenswrapper[2581]:  > containerID="b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996" cmd=["/bin/sleep","15"]
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.175881    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" message="" err=<
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:  >
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:20:06.175898    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]:  > pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerName="main" containerID="cri-o://b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"
Sep 16 16:20:06.176146 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.175947    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerName="main" containerID="cri-o://b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996" gracePeriod=60
Sep 16 16:20:06.526219 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.526192    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:20:06.614803 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.614767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmcsp\" (UniqueName: \"kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.614979 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.614860    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.614979 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.614894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.614979 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.614947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.615136 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.615007    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.615136 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.615047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs\") pod \"d92d1409-3c2b-4353-a7df-3d98da39c125\" (UID: \"d92d1409-3c2b-4353-a7df-3d98da39c125\") "
Sep 16 16:20:06.615306 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.615264    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home" (OuterVolumeSpecName: "home") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:20:06.615306 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.615284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache" (OuterVolumeSpecName: "model-cache") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:20:06.615412 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.615318    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:20:06.617333 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.617298    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:20:06.617333 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.617305    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp" (OuterVolumeSpecName: "kube-api-access-vmcsp") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "kube-api-access-vmcsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:20:06.617517 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.617305    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm" (OuterVolumeSpecName: "dshm") pod "d92d1409-3c2b-4353-a7df-3d98da39c125" (UID: "d92d1409-3c2b-4353-a7df-3d98da39c125"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716488    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716520    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716528    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716555    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92d1409-3c2b-4353-a7df-3d98da39c125-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716566    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92d1409-3c2b-4353-a7df-3d98da39c125-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:06.716581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:06.716574    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmcsp\" (UniqueName: \"kubernetes.io/projected/d92d1409-3c2b-4353-a7df-3d98da39c125-kube-api-access-vmcsp\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:07.243252 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.243218    2581 generic.go:356] "Generic (PLEG): container finished" podID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerID="b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996" exitCode=0
Sep 16 16:20:07.243252 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.243259    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"
Sep 16 16:20:07.243742 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.243289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" event={"ID":"d92d1409-3c2b-4353-a7df-3d98da39c125","Type":"ContainerDied","Data":"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"}
Sep 16 16:20:07.243742 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.243349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r" event={"ID":"d92d1409-3c2b-4353-a7df-3d98da39c125","Type":"ContainerDied","Data":"96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e"}
Sep 16 16:20:07.243742 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.243381    2581 scope.go:117] "RemoveContainer" containerID="b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"
Sep 16 16:20:07.255259 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.255235    2581 scope.go:117] "RemoveContainer" containerID="b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"
Sep 16 16:20:07.255533 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:20:07.255514    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996\": container with ID starting with b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996 not found: ID does not exist" containerID="b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"
Sep 16 16:20:07.255591 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.255543    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996"} err="failed to get container status \"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996\": rpc error: code = NotFound desc = could not find container \"b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996\": container with ID starting with b03f7462a4cacb8308e0aefe63a38b6ce30ea18b879eb315fb171ee20df9e996 not found: ID does not exist"
Sep 16 16:20:07.266671 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.266647    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:20:07.271497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.271474    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-8665687897-r9j6r"]
Sep 16 16:20:07.271748 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:20:07.271724    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92d1409_3c2b_4353_a7df_3d98da39c125.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92d1409_3c2b_4353_a7df_3d98da39c125.slice/crio-96b8ccfac129c44ee052b5cc8acfc03d789f6b4aba4dd480b32d12968b81c30e\": RecentStats: unable to find data in memory cache]"
Sep 16 16:20:07.311187 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:07.311140    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" path="/var/lib/kubelet/pods/d92d1409-3c2b-4353-a7df-3d98da39c125/volumes"
Sep 16 16:20:08.709606 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:08.709571    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:20:18.709703 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:18.709662    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:20:21.195409 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:21.195369    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" containerID="cri-o://ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0" gracePeriod=45
Sep 16 16:20:28.710373 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:28.710327    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:20:28.710767 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:28.710436    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:20:38.710129 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:38.710089    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:20:48.709739 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:48.709702    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:20:51.440294 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.440268    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:20:51.480827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.480742    2581 generic.go:356] "Generic (PLEG): container finished" podID="ae325778-cade-43d0-921e-564a2a4b7e74" containerID="ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0" exitCode=0
Sep 16 16:20:51.480827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.480788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" event={"ID":"ae325778-cade-43d0-921e-564a2a4b7e74","Type":"ContainerDied","Data":"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"}
Sep 16 16:20:51.480827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.480795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"
Sep 16 16:20:51.481061 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.480839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml" event={"ID":"ae325778-cade-43d0-921e-564a2a4b7e74","Type":"ContainerDied","Data":"e517cdcc88b8b0328abfd2e44d0c719d2dc1fed0837457ac7b6270da578d04c6"}
Sep 16 16:20:51.481061 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.480869    2581 scope.go:117] "RemoveContainer" containerID="ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"
Sep 16 16:20:51.487758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.487732    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkbr8\" (UniqueName: \"kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8\") pod \"ae325778-cade-43d0-921e-564a2a4b7e74\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") "
Sep 16 16:20:51.487865 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.487771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs\") pod \"ae325778-cade-43d0-921e-564a2a4b7e74\" (UID: \"ae325778-cade-43d0-921e-564a2a4b7e74\") "
Sep 16 16:20:51.489786 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.489754    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8" (OuterVolumeSpecName: "kube-api-access-fkbr8") pod "ae325778-cade-43d0-921e-564a2a4b7e74" (UID: "ae325778-cade-43d0-921e-564a2a4b7e74"). InnerVolumeSpecName "kube-api-access-fkbr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:20:51.489885 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.489828    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae325778-cade-43d0-921e-564a2a4b7e74" (UID: "ae325778-cade-43d0-921e-564a2a4b7e74"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:20:51.490028 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.490009    2581 scope.go:117] "RemoveContainer" containerID="ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"
Sep 16 16:20:51.490368 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:20:51.490346    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0\": container with ID starting with ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0 not found: ID does not exist" containerID="ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"
Sep 16 16:20:51.490439 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.490374    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0"} err="failed to get container status \"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0\": rpc error: code = NotFound desc = could not find container \"ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0\": container with ID starting with ac77a944077370c4b13262fdb48ae59d4b9bd9fb02b32ec6b31579e1d64a64d0 not found: ID does not exist"
Sep 16 16:20:51.589341 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.589308    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkbr8\" (UniqueName: \"kubernetes.io/projected/ae325778-cade-43d0-921e-564a2a4b7e74-kube-api-access-fkbr8\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:51.589341 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.589337    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae325778-cade-43d0-921e-564a2a4b7e74-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:20:51.803800 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.803769    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:20:51.807767 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:51.807742    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75wnqml"]
Sep 16 16:20:53.309906 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:20:53.309871    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" path="/var/lib/kubelet/pods/ae325778-cade-43d0-921e-564a2a4b7e74/volumes"
Sep 16 16:21:18.374278 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374236    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:21:18.374802 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374782    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerName="main"
Sep 16 16:21:18.374885 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374805    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerName="main"
Sep 16 16:21:18.374885 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374825    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main"
Sep 16 16:21:18.374885 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374833    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main"
Sep 16 16:21:18.375040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374928    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d92d1409-3c2b-4353-a7df-3d98da39c125" containerName="main"
Sep 16 16:21:18.375040 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.374942    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae325778-cade-43d0-921e-564a2a4b7e74" containerName="main"
Sep 16 16:21:18.377799 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.377776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.382184 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.382136    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"openshift-service-ca.crt\""
Sep 16 16:21:18.382375 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.382346    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"canary-v2-kserve-self-signed-certs\""
Sep 16 16:21:18.382515 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.382283    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"default-dockercfg-72nhm\""
Sep 16 16:21:18.382592 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.382206    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"kube-root-ca.crt\""
Sep 16 16:21:18.387797 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.387777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:21:18.526290 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.526461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.526461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.526461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.526461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.526461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.526456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjg74\" (UniqueName: \"kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.626953 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.626867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.626953 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.626905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjg74\" (UniqueName: \"kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.626953 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.626933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.626957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.626975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.627012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627464 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.627433    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627527 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.627501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.627571 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.627539    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.629310 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.629286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.629679 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.629661    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.635609 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.635583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjg74\" (UniqueName: \"kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74\") pod \"canary-v2-kserve-7f65dbbccc-77knv\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.690341 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.690305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:18.969419 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.969379    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:21:18.971778 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:21:18.971746    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965de457_65f8_43a6_9230_594dd1e627da.slice/crio-8800654e7b59d7e0a577b8c1b754688ea3119ee159a0a717d8dc20ac943d31d6 WatchSource:0}: Error finding container 8800654e7b59d7e0a577b8c1b754688ea3119ee159a0a717d8dc20ac943d31d6: Status 404 returned error can't find the container with id 8800654e7b59d7e0a577b8c1b754688ea3119ee159a0a717d8dc20ac943d31d6
Sep 16 16:21:18.974391 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:18.974370    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:21:19.628217 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:19.628180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" event={"ID":"965de457-65f8-43a6-9230-594dd1e627da","Type":"ContainerStarted","Data":"8800654e7b59d7e0a577b8c1b754688ea3119ee159a0a717d8dc20ac943d31d6"}
Sep 16 16:21:20.634896 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:20.634859    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" event={"ID":"965de457-65f8-43a6-9230-594dd1e627da","Type":"ContainerStarted","Data":"6d76fff7d595a9e6d874e0479defcf3ad17938eb8b6a9dcfe79b12705a17ec5e"}
Sep 16 16:21:20.655118 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:20.655063    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" podStartSLOduration=1.591011172 podStartE2EDuration="2.655044663s" totalImagesPullingTime="1.064033491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:21:20.652543557 +0000 UTC m=+1877.880001581" watchObservedRunningTime="2026-09-16 16:21:20.655044663 +0000 UTC m=+1877.882502669"
Sep 16 16:21:20.691220 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:20.691149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:28.691110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:28.691016    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:28.703586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:28.703559    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:29.694245 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:29.694216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:21:47.040098 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.040065    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:21:47.043484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.043467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.047511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.047485    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Sep 16 16:21:47.047633 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.047555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Sep 16 16:21:47.047633 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.047598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-mcfhc\""
Sep 16 16:21:47.047726 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.047711    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Sep 16 16:21:47.047767 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.047726    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-6qvmv\""
Sep 16 16:21:47.055166 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.055129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:21:47.073953 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.073923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.074091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.073975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjw78\" (UniqueName: \"kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.174524 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.174477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjw78\" (UniqueName: \"kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.174701 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.174585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.177119 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.177099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.184950 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.184926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjw78\" (UniqueName: \"kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched2f66h\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.354135 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.354044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.629991 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.629960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:21:47.631905 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:21:47.631879    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99704ea_e547_4af2_b098_c066414d0370.slice/crio-a9e829691e3b53f44c1a9e011616618e0872060874f0da433f1f5db90b45e7ab WatchSource:0}: Error finding container a9e829691e3b53f44c1a9e011616618e0872060874f0da433f1f5db90b45e7ab: Status 404 returned error can't find the container with id a9e829691e3b53f44c1a9e011616618e0872060874f0da433f1f5db90b45e7ab
Sep 16 16:21:47.779968 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.779934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" event={"ID":"d99704ea-e547-4af2-b098-c066414d0370","Type":"ContainerStarted","Data":"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"}
Sep 16 16:21:47.780198 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.779997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" event={"ID":"d99704ea-e547-4af2-b098-c066414d0370","Type":"ContainerStarted","Data":"a9e829691e3b53f44c1a9e011616618e0872060874f0da433f1f5db90b45e7ab"}
Sep 16 16:21:47.780198 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.780027    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:21:47.800015 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:21:47.799965    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podStartSLOduration=0.799948968 podStartE2EDuration="799.948968ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:21:47.797895613 +0000 UTC m=+1905.025353759" watchObservedRunningTime="2026-09-16 16:21:47.799948968 +0000 UTC m=+1905.027406974"
Sep 16 16:22:18.787455 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:18.787423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:22:20.673499 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:20.673448    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:22:28.787269 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:28.787226    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:22:35.693066 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:35.693022    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main" containerID="cri-o://a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707" gracePeriod=45
Sep 16 16:22:38.787815 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:38.787777    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:22:48.787658 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:48.787619    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:22:48.788091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:48.787722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:22:58.787318 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:22:58.787232    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:23:05.943487 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:05.943458    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:23:06.057613 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.057516    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs\") pod \"d99704ea-e547-4af2-b098-c066414d0370\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") "
Sep 16 16:23:06.057613 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.057555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjw78\" (UniqueName: \"kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78\") pod \"d99704ea-e547-4af2-b098-c066414d0370\" (UID: \"d99704ea-e547-4af2-b098-c066414d0370\") "
Sep 16 16:23:06.059867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.059827    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78" (OuterVolumeSpecName: "kube-api-access-mjw78") pod "d99704ea-e547-4af2-b098-c066414d0370" (UID: "d99704ea-e547-4af2-b098-c066414d0370"). InnerVolumeSpecName "kube-api-access-mjw78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:23:06.059867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.059851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d99704ea-e547-4af2-b098-c066414d0370" (UID: "d99704ea-e547-4af2-b098-c066414d0370"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:23:06.158856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.158813    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d99704ea-e547-4af2-b098-c066414d0370-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:23:06.158856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.158858    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjw78\" (UniqueName: \"kubernetes.io/projected/d99704ea-e547-4af2-b098-c066414d0370-kube-api-access-mjw78\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:23:06.198775 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.198740    2581 generic.go:356] "Generic (PLEG): container finished" podID="d99704ea-e547-4af2-b098-c066414d0370" containerID="a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707" exitCode=0
Sep 16 16:23:06.198920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.198785    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"
Sep 16 16:23:06.198920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.198825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" event={"ID":"d99704ea-e547-4af2-b098-c066414d0370","Type":"ContainerDied","Data":"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"}
Sep 16 16:23:06.198920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.198881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h" event={"ID":"d99704ea-e547-4af2-b098-c066414d0370","Type":"ContainerDied","Data":"a9e829691e3b53f44c1a9e011616618e0872060874f0da433f1f5db90b45e7ab"}
Sep 16 16:23:06.198920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.198909    2581 scope.go:117] "RemoveContainer" containerID="a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"
Sep 16 16:23:06.207894 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.207878    2581 scope.go:117] "RemoveContainer" containerID="a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"
Sep 16 16:23:06.208237 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:23:06.208145    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707\": container with ID starting with a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707 not found: ID does not exist" containerID="a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"
Sep 16 16:23:06.208303 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.208247    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707"} err="failed to get container status \"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707\": rpc error: code = NotFound desc = could not find container \"a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707\": container with ID starting with a01efbe5deab4b5f42ac8d7c2307e5a23a1a7b3a2a8e7c6e0d6847bf4e34f707 not found: ID does not exist"
Sep 16 16:23:06.220963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.220939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:23:06.224745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:06.224724    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-sched2f66h"]
Sep 16 16:23:07.310431 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:07.310396    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99704ea-e547-4af2-b098-c066414d0370" path="/var/lib/kubelet/pods/d99704ea-e547-4af2-b098-c066414d0370/volumes"
Sep 16 16:23:22.544037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.543963    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:23:22.545041 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.545015    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main"
Sep 16 16:23:22.545041 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.545039    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main"
Sep 16 16:23:22.545209 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.545108    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d99704ea-e547-4af2-b098-c066414d0370" containerName="main"
Sep 16 16:23:22.548363 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.548343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.552881 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.552728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Sep 16 16:23:22.552881 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.552758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-qvcct\""
Sep 16 16:23:22.552881 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.552758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-gfmsr\""
Sep 16 16:23:22.553192 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.552921    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Sep 16 16:23:22.553192 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.553066    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Sep 16 16:23:22.559636 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.559605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:23:22.710530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.710485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmlmq\" (UniqueName: \"kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.710719 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.710619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.811243 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.811148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.811243 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.811234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmlmq\" (UniqueName: \"kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.813929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.813899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.819725 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.819694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmlmq\" (UniqueName: \"kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:22.860402 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:22.860361    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:23.134054 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:23.134022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:23:23.134904 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:23:23.134878    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725b417e_e8e0_489c_a910_c8dfa8a395e0.slice/crio-b8470bf84906c1c2a0de447552d155d0411dd42ca3610d88d0dc9eda565953fa WatchSource:0}: Error finding container b8470bf84906c1c2a0de447552d155d0411dd42ca3610d88d0dc9eda565953fa: Status 404 returned error can't find the container with id b8470bf84906c1c2a0de447552d155d0411dd42ca3610d88d0dc9eda565953fa
Sep 16 16:23:23.296777 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:23.296735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" event={"ID":"725b417e-e8e0-489c-a910-c8dfa8a395e0","Type":"ContainerStarted","Data":"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"}
Sep 16 16:23:23.296982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:23.296896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:23:23.296982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:23.296952    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" event={"ID":"725b417e-e8e0-489c-a910-c8dfa8a395e0","Type":"ContainerStarted","Data":"b8470bf84906c1c2a0de447552d155d0411dd42ca3610d88d0dc9eda565953fa"}
Sep 16 16:23:23.315402 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:23.315349    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podStartSLOduration=1.31533436 podStartE2EDuration="1.31533436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:23:23.31266465 +0000 UTC m=+2000.540122652" watchObservedRunningTime="2026-09-16 16:23:23.31533436 +0000 UTC m=+2000.542792366"
Sep 16 16:23:51.392970 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:51.392931    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:23:54.304098 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:23:54.304068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:24:06.418390 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.418334    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" podUID="965de457-65f8-43a6-9230-594dd1e627da" containerName="main" containerID="cri-o://6d76fff7d595a9e6d874e0479defcf3ad17938eb8b6a9dcfe79b12705a17ec5e" gracePeriod=45
Sep 16 16:24:06.534323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.534291    2581 generic.go:356] "Generic (PLEG): container finished" podID="965de457-65f8-43a6-9230-594dd1e627da" containerID="6d76fff7d595a9e6d874e0479defcf3ad17938eb8b6a9dcfe79b12705a17ec5e" exitCode=0
Sep 16 16:24:06.534494 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.534356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" event={"ID":"965de457-65f8-43a6-9230-594dd1e627da","Type":"ContainerDied","Data":"6d76fff7d595a9e6d874e0479defcf3ad17938eb8b6a9dcfe79b12705a17ec5e"}
Sep 16 16:24:06.661674 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.661647    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:24:06.807997 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.807915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.807997 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.807950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.808287 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808003    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.808287 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808024    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.808287 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808059    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.808287 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808100    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjg74\" (UniqueName: \"kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74\") pod \"965de457-65f8-43a6-9230-594dd1e627da\" (UID: \"965de457-65f8-43a6-9230-594dd1e627da\") "
Sep 16 16:24:06.808480 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808322    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:24:06.808480 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache" (OuterVolumeSpecName: "model-cache") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:24:06.808480 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.808231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home" (OuterVolumeSpecName: "home") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:24:06.810371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.810342    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm" (OuterVolumeSpecName: "dshm") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:24:06.810371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.810361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74" (OuterVolumeSpecName: "kube-api-access-bjg74") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "kube-api-access-bjg74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:24:06.810371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.810347    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "965de457-65f8-43a6-9230-594dd1e627da" (UID: "965de457-65f8-43a6-9230-594dd1e627da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:24:06.909384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909347    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:06.909384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909383    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjg74\" (UniqueName: \"kubernetes.io/projected/965de457-65f8-43a6-9230-594dd1e627da-kube-api-access-bjg74\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:06.909384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909393    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:06.909629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909402    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:06.909629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909412    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965de457-65f8-43a6-9230-594dd1e627da-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:06.909629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:06.909419    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965de457-65f8-43a6-9230-594dd1e627da-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:24:07.541096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:07.541067    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"
Sep 16 16:24:07.541518 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:07.541099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv" event={"ID":"965de457-65f8-43a6-9230-594dd1e627da","Type":"ContainerDied","Data":"8800654e7b59d7e0a577b8c1b754688ea3119ee159a0a717d8dc20ac943d31d6"}
Sep 16 16:24:07.541518 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:07.541187    2581 scope.go:117] "RemoveContainer" containerID="6d76fff7d595a9e6d874e0479defcf3ad17938eb8b6a9dcfe79b12705a17ec5e"
Sep 16 16:24:07.560004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:07.559980    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:24:07.564182 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:07.564147    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v2-kserve-7f65dbbccc-77knv"]
Sep 16 16:24:09.310205 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:09.310149    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="965de457-65f8-43a6-9230-594dd1e627da" path="/var/lib/kubelet/pods/965de457-65f8-43a6-9230-594dd1e627da/volumes"
Sep 16 16:24:26.762696 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.762618    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:24:26.763143 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.762982    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="965de457-65f8-43a6-9230-594dd1e627da" containerName="main"
Sep 16 16:24:26.763143 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.762994    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="965de457-65f8-43a6-9230-594dd1e627da" containerName="main"
Sep 16 16:24:26.763143 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.763064    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="965de457-65f8-43a6-9230-594dd1e627da" containerName="main"
Sep 16 16:24:26.767516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.767498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.770328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.770298    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"openshift-service-ca.crt\""
Sep 16 16:24:26.770480 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.770364    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"kube-root-ca.crt\""
Sep 16 16:24:26.771716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.771697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"default-dockercfg-67wfp\""
Sep 16 16:24:26.771824 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.771703    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"test-div-v2-kserve-self-signed-certs\""
Sep 16 16:24:26.777319 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.777297    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:24:26.871691 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.871890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mkh6\" (UniqueName: \"kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.871890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.871890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.871890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.871890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.871857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973039 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mkh6\" (UniqueName: \"kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973039 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973338 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973338 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973338 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973338 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973599 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973665 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.973665 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.973647    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.975499 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.975473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.975713 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.975695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:26.981248 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:26.981225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mkh6\" (UniqueName: \"kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6\") pod \"test-div-v2-kserve-5f6fdb6b77-lmlwh\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:27.080922 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:27.080833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:27.362890 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:27.362865    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:24:27.365597 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:24:27.365565    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f8ed40_0010_474a_a878_d49cbbac27e5.slice/crio-f2d778bb529c29e207be00d0264ff21f5b57e6c0b6b18926356ea4426ecbc3b1 WatchSource:0}: Error finding container f2d778bb529c29e207be00d0264ff21f5b57e6c0b6b18926356ea4426ecbc3b1: Status 404 returned error can't find the container with id f2d778bb529c29e207be00d0264ff21f5b57e6c0b6b18926356ea4426ecbc3b1
Sep 16 16:24:27.651537 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:27.651466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" event={"ID":"f2f8ed40-0010-474a-a878-d49cbbac27e5","Type":"ContainerStarted","Data":"5b0cdca2081f8b7f6a7052791ec366a86591aea76f74f260cf5d0464a8f140b7"}
Sep 16 16:24:27.651537 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:27.651543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" event={"ID":"f2f8ed40-0010-474a-a878-d49cbbac27e5","Type":"ContainerStarted","Data":"f2d778bb529c29e207be00d0264ff21f5b57e6c0b6b18926356ea4426ecbc3b1"}
Sep 16 16:24:27.674282 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:27.674226    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" podStartSLOduration=1.674208106 podStartE2EDuration="1.674208106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:24:27.671939983 +0000 UTC m=+2064.899397991" watchObservedRunningTime="2026-09-16 16:24:27.674208106 +0000 UTC m=+2064.901666112"
Sep 16 16:24:28.081735 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:28.081655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:37.081197 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:37.081139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:37.093965 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:37.093932    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:37.715391 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:37.715356    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:24:51.641107 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:51.641071    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:24:53.667516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:53.667476    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:24:54.304574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:24:54.304521    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:25:03.434667 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:03.434638    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:25:03.436437 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:03.436412    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:25:03.440364 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:03.440343    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:25:03.442207 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:03.442188    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:25:04.304823 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:04.304775    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:25:06.664310 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:06.664268    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" containerID="cri-o://d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4" gracePeriod=45
Sep 16 16:25:08.693645 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:08.693561    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerName="main" containerID="cri-o://5b0cdca2081f8b7f6a7052791ec366a86591aea76f74f260cf5d0464a8f140b7" gracePeriod=45
Sep 16 16:25:08.704464 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:08.704424    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": dial tcp 10.133.0.62:8000: connect: connection refused"
Sep 16 16:25:08.880650 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:08.880614    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerID="5b0cdca2081f8b7f6a7052791ec366a86591aea76f74f260cf5d0464a8f140b7" exitCode=0
Sep 16 16:25:08.880822 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:08.880656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" event={"ID":"f2f8ed40-0010-474a-a878-d49cbbac27e5","Type":"ContainerDied","Data":"5b0cdca2081f8b7f6a7052791ec366a86591aea76f74f260cf5d0464a8f140b7"}
Sep 16 16:25:08.954021 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:08.953963    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:25:09.051558 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.051760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.051760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051613    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mkh6\" (UniqueName: \"kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.051760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051627    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.051760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051661    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.051760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051690    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm\") pod \"f2f8ed40-0010-474a-a878-d49cbbac27e5\" (UID: \"f2f8ed40-0010-474a-a878-d49cbbac27e5\") "
Sep 16 16:25:09.052018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:25:09.052018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051956    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home" (OuterVolumeSpecName: "home") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:25:09.052018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.051971    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:25:09.054056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.054029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6" (OuterVolumeSpecName: "kube-api-access-6mkh6") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "kube-api-access-6mkh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:25:09.054202 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.054066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:25:09.054202 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.054077    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm" (OuterVolumeSpecName: "dshm") pod "f2f8ed40-0010-474a-a878-d49cbbac27e5" (UID: "f2f8ed40-0010-474a-a878-d49cbbac27e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:25:09.152867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152826    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.152867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152859    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.152867 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152870    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.153092 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152879    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mkh6\" (UniqueName: \"kubernetes.io/projected/f2f8ed40-0010-474a-a878-d49cbbac27e5-kube-api-access-6mkh6\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.153092 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152889    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2f8ed40-0010-474a-a878-d49cbbac27e5-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.153092 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.152897    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2f8ed40-0010-474a-a878-d49cbbac27e5-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:09.888456 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.888419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh" event={"ID":"f2f8ed40-0010-474a-a878-d49cbbac27e5","Type":"ContainerDied","Data":"f2d778bb529c29e207be00d0264ff21f5b57e6c0b6b18926356ea4426ecbc3b1"}
Sep 16 16:25:09.888456 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.888453    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"
Sep 16 16:25:09.888915 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.888493    2581 scope.go:117] "RemoveContainer" containerID="5b0cdca2081f8b7f6a7052791ec366a86591aea76f74f260cf5d0464a8f140b7"
Sep 16 16:25:09.905856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.905827    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:25:09.909098 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:09.909070    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-lmlwh"]
Sep 16 16:25:11.310783 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:11.310745    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" path="/var/lib/kubelet/pods/f2f8ed40-0010-474a-a878-d49cbbac27e5/volumes"
Sep 16 16:25:14.304660 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:14.304620    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:25:14.305118 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:14.304780    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:25:24.304723 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:24.304683    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:25:34.305067 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:34.305026    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:25:36.921139 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:36.921072    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:25:37.008550 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.008517    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs\") pod \"725b417e-e8e0-489c-a910-c8dfa8a395e0\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") "
Sep 16 16:25:37.008726 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.008645    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmlmq\" (UniqueName: \"kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq\") pod \"725b417e-e8e0-489c-a910-c8dfa8a395e0\" (UID: \"725b417e-e8e0-489c-a910-c8dfa8a395e0\") "
Sep 16 16:25:37.010800 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.010770    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "725b417e-e8e0-489c-a910-c8dfa8a395e0" (UID: "725b417e-e8e0-489c-a910-c8dfa8a395e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:25:37.010929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.010819    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq" (OuterVolumeSpecName: "kube-api-access-tmlmq") pod "725b417e-e8e0-489c-a910-c8dfa8a395e0" (UID: "725b417e-e8e0-489c-a910-c8dfa8a395e0"). InnerVolumeSpecName "kube-api-access-tmlmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:25:37.031048 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.031016    2581 generic.go:356] "Generic (PLEG): container finished" podID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerID="d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4" exitCode=0
Sep 16 16:25:37.031190 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.031067    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"
Sep 16 16:25:37.031190 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.031089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" event={"ID":"725b417e-e8e0-489c-a910-c8dfa8a395e0","Type":"ContainerDied","Data":"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"}
Sep 16 16:25:37.031190 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.031141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh" event={"ID":"725b417e-e8e0-489c-a910-c8dfa8a395e0","Type":"ContainerDied","Data":"b8470bf84906c1c2a0de447552d155d0411dd42ca3610d88d0dc9eda565953fa"}
Sep 16 16:25:37.031304 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.031197    2581 scope.go:117] "RemoveContainer" containerID="d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"
Sep 16 16:25:37.050584 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.050535    2581 scope.go:117] "RemoveContainer" containerID="d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"
Sep 16 16:25:37.050895 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:25:37.050873    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4\": container with ID starting with d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4 not found: ID does not exist" containerID="d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"
Sep 16 16:25:37.050946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.050906    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4"} err="failed to get container status \"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4\": rpc error: code = NotFound desc = could not find container \"d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4\": container with ID starting with d0c29a9e901b820b7d842ed8e049a64f24658e27aa2e69811a45e7f3a8d116c4 not found: ID does not exist"
Sep 16 16:25:37.064212 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.064181    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:25:37.069534 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.069510    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b69g6bdh"]
Sep 16 16:25:37.109856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.109809    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmlmq\" (UniqueName: \"kubernetes.io/projected/725b417e-e8e0-489c-a910-c8dfa8a395e0-kube-api-access-tmlmq\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:37.109856 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.109848    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725b417e-e8e0-489c-a910-c8dfa8a395e0-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:25:37.309857 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:37.309775    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" path="/var/lib/kubelet/pods/725b417e-e8e0-489c-a910-c8dfa8a395e0/volumes"
Sep 16 16:25:47.665236 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665201    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:25:47.665718 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665699    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main"
Sep 16 16:25:47.665792 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665723    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main"
Sep 16 16:25:47.665792 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665747    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerName="main"
Sep 16 16:25:47.665792 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665755    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerName="main"
Sep 16 16:25:47.665944 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665835    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f8ed40-0010-474a-a878-d49cbbac27e5" containerName="main"
Sep 16 16:25:47.665944 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.665853    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="725b417e-e8e0-489c-a910-c8dfa8a395e0" containerName="main"
Sep 16 16:25:47.670171 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.670141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.673081 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.673052    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"openshift-service-ca.crt\""
Sep 16 16:25:47.674421 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.674398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"default-dockercfg-bqzw6\""
Sep 16 16:25:47.674556 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.674407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"kube-root-ca.crt\""
Sep 16 16:25:47.674556 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.674407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"stop-v1-kserve-self-signed-certs\""
Sep 16 16:25:47.678676 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.678650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:25:47.700788 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.700917 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.700917 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.700917 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.700917 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tdn7\" (UniqueName: \"kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.700917 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.700893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.801785 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tdn7\" (UniqueName: \"kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802004 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.801904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802390 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.802269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802390 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.802298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.802390 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.802326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.804201 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.804178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.804385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.804367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.810903 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.810879    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tdn7\" (UniqueName: \"kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7\") pod \"stop-v1-kserve-5657b88c95-mx4kl\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") " pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:47.982427 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:47.982327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:48.289639 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:48.289531    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:25:48.295813 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:25:48.295778    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191d4127_79ab_46f6_a6b0_a4091a9ddc99.slice/crio-fa1898d3ba607f429e41ab4909abc2f02909db55c6b86f88801e7e6e4091a7d0 WatchSource:0}: Error finding container fa1898d3ba607f429e41ab4909abc2f02909db55c6b86f88801e7e6e4091a7d0: Status 404 returned error can't find the container with id fa1898d3ba607f429e41ab4909abc2f02909db55c6b86f88801e7e6e4091a7d0
Sep 16 16:25:49.098711 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:49.098667    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" event={"ID":"191d4127-79ab-46f6-a6b0-a4091a9ddc99","Type":"ContainerStarted","Data":"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"}
Sep 16 16:25:49.099222 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:49.098763    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" event={"ID":"191d4127-79ab-46f6-a6b0-a4091a9ddc99","Type":"ContainerStarted","Data":"fa1898d3ba607f429e41ab4909abc2f02909db55c6b86f88801e7e6e4091a7d0"}
Sep 16 16:25:49.120479 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:49.120431    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" podStartSLOduration=2.120416894 podStartE2EDuration="2.120416894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:25:49.117719069 +0000 UTC m=+2146.345177077" watchObservedRunningTime="2026-09-16 16:25:49.120416894 +0000 UTC m=+2146.347874900"
Sep 16 16:25:49.983332 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:49.983290    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:52.564110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.564074    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:25:52.567601 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.567580    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.570723 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.570697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-3e394e04\"/\"stop-v2-kserve-self-signed-certs\""
Sep 16 16:25:52.579575 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.579544    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:25:52.640096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.640096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.640344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.640344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.640344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnp2f\" (UniqueName: \"kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.640344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.640300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741399 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741399 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741629 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnp2f\" (UniqueName: \"kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741738 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741846 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741824    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741913 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.741976 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.741934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.743938 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.743908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.744123 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.744107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.749563 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.749531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnp2f\" (UniqueName: \"kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f\") pod \"stop-v2-kserve-7f68747c45-7459l\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") " pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:52.882284 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:52.882244    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:53.160358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:53.160322    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:25:53.162243 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:25:53.162205    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086291c8_864a_42a0_ab55_14620c478717.slice/crio-64bd1ece1370454b56700e8a1e3c2342f3b95683209692425c57f791bc5d95a2 WatchSource:0}: Error finding container 64bd1ece1370454b56700e8a1e3c2342f3b95683209692425c57f791bc5d95a2: Status 404 returned error can't find the container with id 64bd1ece1370454b56700e8a1e3c2342f3b95683209692425c57f791bc5d95a2
Sep 16 16:25:54.128057 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:54.128015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" event={"ID":"086291c8-864a-42a0-ab55-14620c478717","Type":"ContainerStarted","Data":"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"}
Sep 16 16:25:54.128468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:54.128079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" event={"ID":"086291c8-864a-42a0-ab55-14620c478717","Type":"ContainerStarted","Data":"64bd1ece1370454b56700e8a1e3c2342f3b95683209692425c57f791bc5d95a2"}
Sep 16 16:25:54.144916 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:54.144868    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" podStartSLOduration=2.144852122 podStartE2EDuration="2.144852122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:25:54.143508084 +0000 UTC m=+2151.370966092" watchObservedRunningTime="2026-09-16 16:25:54.144852122 +0000 UTC m=+2151.372310128"
Sep 16 16:25:54.883065 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:54.883028    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:25:57.983122 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:57.983079    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:57.995699 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:57.995668    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:25:58.160457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:25:58.160419    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:26:02.883440 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:02.883393    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:26:02.896402 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:02.896376    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:26:03.188315 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:03.188235    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:26:14.542946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.542906    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:26:14.546850 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.546826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.549565 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.549538    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Sep 16 16:26:14.550815 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.550790    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Sep 16 16:26:14.550926 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.550835    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-swnq8\""
Sep 16 16:26:14.550926 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.550866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-nngbn\""
Sep 16 16:26:14.550926 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.550801    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Sep 16 16:26:14.556591 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.556569    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:26:14.641644 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.641592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.641828 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.641656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb722\" (UniqueName: \"kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.742700 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.742658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.742897 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.742710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb722\" (UniqueName: \"kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.745484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.745451    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.750928 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.750902    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb722\" (UniqueName: \"kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:14.858506 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:14.858416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:15.135839 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:15.135813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:26:15.137496 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:26:15.137459    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabad67de_a709_46a2_aa33_a1c784acfa81.slice/crio-c289658386329beb826121a04f10ccaba89629708671976a249db97a62f84295 WatchSource:0}: Error finding container c289658386329beb826121a04f10ccaba89629708671976a249db97a62f84295: Status 404 returned error can't find the container with id c289658386329beb826121a04f10ccaba89629708671976a249db97a62f84295
Sep 16 16:26:15.253098 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:15.253059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" event={"ID":"abad67de-a709-46a2-aa33-a1c784acfa81","Type":"ContainerStarted","Data":"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"}
Sep 16 16:26:15.253327 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:15.253152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" event={"ID":"abad67de-a709-46a2-aa33-a1c784acfa81","Type":"ContainerStarted","Data":"c289658386329beb826121a04f10ccaba89629708671976a249db97a62f84295"}
Sep 16 16:26:15.253327 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:15.253232    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:15.271459 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:15.271398    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podStartSLOduration=1.271376827 podStartE2EDuration="1.271376827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:26:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:26:15.269762989 +0000 UTC m=+2172.497221006" watchObservedRunningTime="2026-09-16 16:26:15.271376827 +0000 UTC m=+2172.498834835"
Sep 16 16:26:46.262270 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:46.262235    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:26:51.315901 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:51.315869    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:26:55.078912 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:55.078868    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:26:56.261584 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:26:56.261538    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:27:06.261516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:06.261471    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:27:06.336736 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:06.336694    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" containerID="cri-o://857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3" gracePeriod=45
Sep 16 16:27:10.101853 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.101806    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" podUID="086291c8-864a-42a0-ab55-14620c478717" containerName="main" containerID="cri-o://871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b" gracePeriod=45
Sep 16 16:27:10.177689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.177648    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" podUID="086291c8-864a-42a0-ab55-14620c478717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": dial tcp 10.133.0.64:8000: connect: connection refused"
Sep 16 16:27:10.352535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.352473    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:27:10.444398 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444453    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnp2f\" (UniqueName: \"kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444564 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444543    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444772 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir\") pod \"086291c8-864a-42a0-ab55-14620c478717\" (UID: \"086291c8-864a-42a0-ab55-14620c478717\") "
Sep 16 16:27:10.444829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444803    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home" (OuterVolumeSpecName: "home") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:10.444902 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache" (OuterVolumeSpecName: "model-cache") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:10.444977 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.444910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:10.446785 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.446762    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:27:10.447037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.447017    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm" (OuterVolumeSpecName: "dshm") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:10.447076 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.447034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f" (OuterVolumeSpecName: "kube-api-access-tnp2f") pod "086291c8-864a-42a0-ab55-14620c478717" (UID: "086291c8-864a-42a0-ab55-14620c478717"). InnerVolumeSpecName "kube-api-access-tnp2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:27:10.545799 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545762    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnp2f\" (UniqueName: \"kubernetes.io/projected/086291c8-864a-42a0-ab55-14620c478717-kube-api-access-tnp2f\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.545799 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545797    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.545799 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545807    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086291c8-864a-42a0-ab55-14620c478717-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.545799 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545815    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.546063 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545823    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.546063 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.545830    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086291c8-864a-42a0-ab55-14620c478717-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:10.559280 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.559239    2581 generic.go:356] "Generic (PLEG): container finished" podID="086291c8-864a-42a0-ab55-14620c478717" containerID="871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b" exitCode=0
Sep 16 16:27:10.559430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.559292    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"
Sep 16 16:27:10.559430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.559306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" event={"ID":"086291c8-864a-42a0-ab55-14620c478717","Type":"ContainerDied","Data":"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"}
Sep 16 16:27:10.559430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.559357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l" event={"ID":"086291c8-864a-42a0-ab55-14620c478717","Type":"ContainerDied","Data":"64bd1ece1370454b56700e8a1e3c2342f3b95683209692425c57f791bc5d95a2"}
Sep 16 16:27:10.559430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.559380    2581 scope.go:117] "RemoveContainer" containerID="871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"
Sep 16 16:27:10.568846 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.568826    2581 scope.go:117] "RemoveContainer" containerID="871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"
Sep 16 16:27:10.569128 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:27:10.569109    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b\": container with ID starting with 871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b not found: ID does not exist" containerID="871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"
Sep 16 16:27:10.569208 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.569136    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b"} err="failed to get container status \"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b\": rpc error: code = NotFound desc = could not find container \"871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b\": container with ID starting with 871a7095a708dd506c0c655f4f23e3d2f1f1ee23b30cb52a20a59a00cd86a95b not found: ID does not exist"
Sep 16 16:27:10.581819 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.581791    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:27:10.588676 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:10.588650    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v2-kserve-7f68747c45-7459l"]
Sep 16 16:27:11.310335 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:11.310298    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="086291c8-864a-42a0-ab55-14620c478717" path="/var/lib/kubelet/pods/086291c8-864a-42a0-ab55-14620c478717/volumes"
Sep 16 16:27:16.261616 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:16.261574    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:27:16.262088 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:16.261685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:27:16.674734 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:16.674699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:27:26.261844 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:26.261755    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:27:31.695535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:31.695486    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" podUID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" containerName="main" containerID="cri-o://ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7" gracePeriod=45
Sep 16 16:27:31.956126 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:31.956050    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:27:32.041798 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041763    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tdn7\" (UniqueName: \"kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.041798 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041800    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.042056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041833    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.042056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041855    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.042056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041893    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.042056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.041933    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs\") pod \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\" (UID: \"191d4127-79ab-46f6-a6b0-a4091a9ddc99\") "
Sep 16 16:27:32.042325 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.042105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache" (OuterVolumeSpecName: "model-cache") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:32.042325 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.042144    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home" (OuterVolumeSpecName: "home") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:32.042424 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.042319    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.042424 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.042342    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.042530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.042459    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:32.044112 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.044083    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7" (OuterVolumeSpecName: "kube-api-access-7tdn7") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "kube-api-access-7tdn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:27:32.044240 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.044119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm" (OuterVolumeSpecName: "dshm") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:27:32.044394 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.044370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "191d4127-79ab-46f6-a6b0-a4091a9ddc99" (UID: "191d4127-79ab-46f6-a6b0-a4091a9ddc99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:27:32.143577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.143543    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tdn7\" (UniqueName: \"kubernetes.io/projected/191d4127-79ab-46f6-a6b0-a4091a9ddc99-kube-api-access-7tdn7\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.143577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.143571    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.143784 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.143599    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.143784 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.143609    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/191d4127-79ab-46f6-a6b0-a4091a9ddc99-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:32.676195 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.676139    2581 generic.go:356] "Generic (PLEG): container finished" podID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" containerID="ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7" exitCode=0
Sep 16 16:27:32.676361 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.676205    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"
Sep 16 16:27:32.676361 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.676202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" event={"ID":"191d4127-79ab-46f6-a6b0-a4091a9ddc99","Type":"ContainerDied","Data":"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"}
Sep 16 16:27:32.676361 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.676335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl" event={"ID":"191d4127-79ab-46f6-a6b0-a4091a9ddc99","Type":"ContainerDied","Data":"fa1898d3ba607f429e41ab4909abc2f02909db55c6b86f88801e7e6e4091a7d0"}
Sep 16 16:27:32.676551 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.676362    2581 scope.go:117] "RemoveContainer" containerID="ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"
Sep 16 16:27:32.686266 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.686246    2581 scope.go:117] "RemoveContainer" containerID="ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"
Sep 16 16:27:32.686570 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:27:32.686541    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7\": container with ID starting with ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7 not found: ID does not exist" containerID="ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"
Sep 16 16:27:32.686653 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.686579    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7"} err="failed to get container status \"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7\": rpc error: code = NotFound desc = could not find container \"ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7\": container with ID starting with ebf2076cd2b21dc1dab18ded145b0ae4d515f4b5a6e206b7045dbde57d0959e7 not found: ID does not exist"
Sep 16 16:27:32.706464 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.706429    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:27:32.707959 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:32.707934    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-3e394e04/stop-v1-kserve-5657b88c95-mx4kl"]
Sep 16 16:27:33.310964 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:33.310928    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" path="/var/lib/kubelet/pods/191d4127-79ab-46f6-a6b0-a4091a9ddc99/volumes"
Sep 16 16:27:36.261384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.261344    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:27:36.593525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.593500    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:27:36.687016 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.686975    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs\") pod \"abad67de-a709-46a2-aa33-a1c784acfa81\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") "
Sep 16 16:27:36.687223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.687031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb722\" (UniqueName: \"kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722\") pod \"abad67de-a709-46a2-aa33-a1c784acfa81\" (UID: \"abad67de-a709-46a2-aa33-a1c784acfa81\") "
Sep 16 16:27:36.689364 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.689339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abad67de-a709-46a2-aa33-a1c784acfa81" (UID: "abad67de-a709-46a2-aa33-a1c784acfa81"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:27:36.689458 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.689375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722" (OuterVolumeSpecName: "kube-api-access-zb722") pod "abad67de-a709-46a2-aa33-a1c784acfa81" (UID: "abad67de-a709-46a2-aa33-a1c784acfa81"). InnerVolumeSpecName "kube-api-access-zb722". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:27:36.703957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.703931    2581 generic.go:356] "Generic (PLEG): container finished" podID="abad67de-a709-46a2-aa33-a1c784acfa81" containerID="857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3" exitCode=0
Sep 16 16:27:36.704074 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.703973    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"
Sep 16 16:27:36.704074 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.704010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" event={"ID":"abad67de-a709-46a2-aa33-a1c784acfa81","Type":"ContainerDied","Data":"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"}
Sep 16 16:27:36.704074 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.704061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4" event={"ID":"abad67de-a709-46a2-aa33-a1c784acfa81","Type":"ContainerDied","Data":"c289658386329beb826121a04f10ccaba89629708671976a249db97a62f84295"}
Sep 16 16:27:36.704252 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.704083    2581 scope.go:117] "RemoveContainer" containerID="857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"
Sep 16 16:27:36.713082 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.713063    2581 scope.go:117] "RemoveContainer" containerID="857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"
Sep 16 16:27:36.713388 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:27:36.713366    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3\": container with ID starting with 857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3 not found: ID does not exist" containerID="857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"
Sep 16 16:27:36.713484 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.713393    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3"} err="failed to get container status \"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3\": rpc error: code = NotFound desc = could not find container \"857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3\": container with ID starting with 857f4fb1ffc6d09f69bbe386237fac5759c1ac5d7580018df70e239c985080f3 not found: ID does not exist"
Sep 16 16:27:36.726599 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.726571    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:27:36.730610 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.730588    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785qw9p4"]
Sep 16 16:27:36.788284 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.788242    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abad67de-a709-46a2-aa33-a1c784acfa81-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:36.788284 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:36.788274    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb722\" (UniqueName: \"kubernetes.io/projected/abad67de-a709-46a2-aa33-a1c784acfa81-kube-api-access-zb722\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:27:37.311261 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:37.311227    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" path="/var/lib/kubelet/pods/abad67de-a709-46a2-aa33-a1c784acfa81/volumes"
Sep 16 16:27:44.668318 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.668280    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:27:44.669005 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.668980    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main"
Sep 16 16:27:44.669105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669007    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main"
Sep 16 16:27:44.669105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669027    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" containerName="main"
Sep 16 16:27:44.669105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669034    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" containerName="main"
Sep 16 16:27:44.669105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669055    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086291c8-864a-42a0-ab55-14620c478717" containerName="main"
Sep 16 16:27:44.669105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669064    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="086291c8-864a-42a0-ab55-14620c478717" containerName="main"
Sep 16 16:27:44.669392 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669180    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="086291c8-864a-42a0-ab55-14620c478717" containerName="main"
Sep 16 16:27:44.669392 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669194    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="191d4127-79ab-46f6-a6b0-a4091a9ddc99" containerName="main"
Sep 16 16:27:44.669392 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.669206    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="abad67de-a709-46a2-aa33-a1c784acfa81" containerName="main"
Sep 16 16:27:44.672307 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.672286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.676129 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.676106    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-decommission-203cd6d1\"/\"decom-v2-kserve-self-signed-certs\""
Sep 16 16:27:44.676247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.676153    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-decommission-203cd6d1\"/\"kube-root-ca.crt\""
Sep 16 16:27:44.676247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.676185    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-decommission-203cd6d1\"/\"default-dockercfg-6x7fj\""
Sep 16 16:27:44.676247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.676213    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-decommission-203cd6d1\"/\"openshift-service-ca.crt\""
Sep 16 16:27:44.685395 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.685364    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:27:44.763386 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.763557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk98m\" (UniqueName: \"kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.763557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.763557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.763557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.763711 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.763557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864684 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864684 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.864946 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.864917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk98m\" (UniqueName: \"kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.865281 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.865251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.865413 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.865266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.865413 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.865316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.867277 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.867247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.867427 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.867410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.872790 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.872766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk98m\" (UniqueName: \"kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m\") pod \"decom-v2-kserve-77f867d787-jdlwg\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") " pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:44.987408 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:44.987309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:45.263359 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:45.263325    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:27:45.757342 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:45.757298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" event={"ID":"3374c283-91f4-4131-b3f7-a4e7b7615576","Type":"ContainerStarted","Data":"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"}
Sep 16 16:27:45.757741 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:45.757361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" event={"ID":"3374c283-91f4-4131-b3f7-a4e7b7615576","Type":"ContainerStarted","Data":"a2b9923a8437ab029a9a23694a8f403b18fddff73829aaabec72485b7b946045"}
Sep 16 16:27:45.774979 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:45.774928    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" podStartSLOduration=1.774913178 podStartE2EDuration="1.774913178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:27:45.773349978 +0000 UTC m=+2263.000807986" watchObservedRunningTime="2026-09-16 16:27:45.774913178 +0000 UTC m=+2263.002371184"
Sep 16 16:27:45.987601 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:45.987564    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:54.987633 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:54.987579    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:55.000967 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:55.000936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:27:55.827249 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:27:55.827220    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:28:06.967656 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.967617    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:28:06.973018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.972991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:06.977317 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.977284    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-ggv9g\""
Sep 16 16:28:06.977468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.977330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Sep 16 16:28:06.977468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.977347    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Sep 16 16:28:06.977468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.977294    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-qqx6s\""
Sep 16 16:28:06.977468 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.977380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Sep 16 16:28:06.983327 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:06.983303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:28:07.069529 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.069485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdztn\" (UniqueName: \"kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.069710 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.069589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.170529 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.170478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.170717 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.170568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdztn\" (UniqueName: \"kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.173237 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.173212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.180580 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.180553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdztn\" (UniqueName: \"kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.285199 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.285078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.569969 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.569882    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:28:07.573811 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:28:07.573783    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472ffc23_a25e_494a_ac62_2e16a3053352.slice/crio-89dfe16f9b182043ae41e72f4237914c8fe7a550e7625a490c018ee311fc9be7 WatchSource:0}: Error finding container 89dfe16f9b182043ae41e72f4237914c8fe7a550e7625a490c018ee311fc9be7: Status 404 returned error can't find the container with id 89dfe16f9b182043ae41e72f4237914c8fe7a550e7625a490c018ee311fc9be7
Sep 16 16:28:07.886557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.886492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" event={"ID":"472ffc23-a25e-494a-ac62-2e16a3053352","Type":"ContainerStarted","Data":"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"}
Sep 16 16:28:07.886789 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.886575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" event={"ID":"472ffc23-a25e-494a-ac62-2e16a3053352","Type":"ContainerStarted","Data":"89dfe16f9b182043ae41e72f4237914c8fe7a550e7625a490c018ee311fc9be7"}
Sep 16 16:28:07.886789 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.886717    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:07.903773 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:07.903719    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podStartSLOduration=1.903704652 podStartE2EDuration="1.903704652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:28:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:28:07.902978003 +0000 UTC m=+2285.130436011" watchObservedRunningTime="2026-09-16 16:28:07.903704652 +0000 UTC m=+2285.131162658"
Sep 16 16:28:34.787180 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:34.787133    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:28:38.893715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:38.893680    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:28:44.710043 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:44.710003    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:28:48.892840 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:48.892803    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:28:49.806457 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:49.806411    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerName="main" containerID="cri-o://ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe" gracePeriod=45
Sep 16 16:28:49.817280 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:49.817242    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": dial tcp 10.133.0.66:8000: connect: connection refused"
Sep 16 16:28:50.072296 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.071895    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:28:50.125534 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.125493    2581 generic.go:356] "Generic (PLEG): container finished" podID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerID="ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe" exitCode=0
Sep 16 16:28:50.125716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.125545    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"
Sep 16 16:28:50.125716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.125575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" event={"ID":"3374c283-91f4-4131-b3f7-a4e7b7615576","Type":"ContainerDied","Data":"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"}
Sep 16 16:28:50.125716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.125635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg" event={"ID":"3374c283-91f4-4131-b3f7-a4e7b7615576","Type":"ContainerDied","Data":"a2b9923a8437ab029a9a23694a8f403b18fddff73829aaabec72485b7b946045"}
Sep 16 16:28:50.125716 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.125668    2581 scope.go:117] "RemoveContainer" containerID="ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"
Sep 16 16:28:50.135951 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.135931    2581 scope.go:117] "RemoveContainer" containerID="ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"
Sep 16 16:28:50.136325 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:28:50.136300    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe\": container with ID starting with ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe not found: ID does not exist" containerID="ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"
Sep 16 16:28:50.136433 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.136337    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe"} err="failed to get container status \"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe\": rpc error: code = NotFound desc = could not find container \"ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe\": container with ID starting with ba0141943c108c6f10b42e710634894e0f42a709db4e3aea3f4f6e9d2d8833fe not found: ID does not exist"
Sep 16 16:28:50.159692 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.159871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.159871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk98m\" (UniqueName: \"kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.159871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159768    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.159871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.159871 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159859    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir\") pod \"3374c283-91f4-4131-b3f7-a4e7b7615576\" (UID: \"3374c283-91f4-4131-b3f7-a4e7b7615576\") "
Sep 16 16:28:50.160141 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.159986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache" (OuterVolumeSpecName: "model-cache") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:28:50.160141 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.160076    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home" (OuterVolumeSpecName: "home") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:28:50.160274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.160130    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:28:50.160274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.160265    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.160378 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.160283    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.160378 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.160298    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.162228 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.162201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm" (OuterVolumeSpecName: "dshm") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:28:50.162370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.162235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m" (OuterVolumeSpecName: "kube-api-access-gk98m") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "kube-api-access-gk98m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:28:50.162430 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.162358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3374c283-91f4-4131-b3f7-a4e7b7615576" (UID: "3374c283-91f4-4131-b3f7-a4e7b7615576"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:28:50.261540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.261495    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3374c283-91f4-4131-b3f7-a4e7b7615576-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.261540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.261533    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk98m\" (UniqueName: \"kubernetes.io/projected/3374c283-91f4-4131-b3f7-a4e7b7615576-kube-api-access-gk98m\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.261540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.261545    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3374c283-91f4-4131-b3f7-a4e7b7615576-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:28:50.450549 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.450513    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:28:50.455502 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:50.455470    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-decommission-203cd6d1/decom-v2-kserve-77f867d787-jdlwg"]
Sep 16 16:28:51.311168 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:51.311119    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" path="/var/lib/kubelet/pods/3374c283-91f4-4131-b3f7-a4e7b7615576/volumes"
Sep 16 16:28:58.893765 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:58.893685    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:28:59.731494 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:28:59.731452    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" containerID="cri-o://6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b" gracePeriod=45
Sep 16 16:29:08.893385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:08.893348    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:29:08.893983 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:08.893471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:29:18.893402 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:18.893360    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:29:28.893432 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:28.893386    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:29:29.985209 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:29.985181    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:29:30.112875 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.112767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdztn\" (UniqueName: \"kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn\") pod \"472ffc23-a25e-494a-ac62-2e16a3053352\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") "
Sep 16 16:29:30.112875 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.112820    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs\") pod \"472ffc23-a25e-494a-ac62-2e16a3053352\" (UID: \"472ffc23-a25e-494a-ac62-2e16a3053352\") "
Sep 16 16:29:30.115001 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.114970    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "472ffc23-a25e-494a-ac62-2e16a3053352" (UID: "472ffc23-a25e-494a-ac62-2e16a3053352"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:29:30.115113 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.115041    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn" (OuterVolumeSpecName: "kube-api-access-qdztn") pod "472ffc23-a25e-494a-ac62-2e16a3053352" (UID: "472ffc23-a25e-494a-ac62-2e16a3053352"). InnerVolumeSpecName "kube-api-access-qdztn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:29:30.214211 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.214175    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/472ffc23-a25e-494a-ac62-2e16a3053352-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:29:30.214211 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.214204    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdztn\" (UniqueName: \"kubernetes.io/projected/472ffc23-a25e-494a-ac62-2e16a3053352-kube-api-access-qdztn\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:29:30.347758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.347718    2581 generic.go:356] "Generic (PLEG): container finished" podID="472ffc23-a25e-494a-ac62-2e16a3053352" containerID="6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b" exitCode=0
Sep 16 16:29:30.347758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.347751    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"
Sep 16 16:29:30.348002 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.347798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" event={"ID":"472ffc23-a25e-494a-ac62-2e16a3053352","Type":"ContainerDied","Data":"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"}
Sep 16 16:29:30.348002 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.347846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf" event={"ID":"472ffc23-a25e-494a-ac62-2e16a3053352","Type":"ContainerDied","Data":"89dfe16f9b182043ae41e72f4237914c8fe7a550e7625a490c018ee311fc9be7"}
Sep 16 16:29:30.348002 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.347869    2581 scope.go:117] "RemoveContainer" containerID="6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"
Sep 16 16:29:30.357467 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.357440    2581 scope.go:117] "RemoveContainer" containerID="6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"
Sep 16 16:29:30.357752 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:29:30.357736    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b\": container with ID starting with 6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b not found: ID does not exist" containerID="6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"
Sep 16 16:29:30.357798 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.357760    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b"} err="failed to get container status \"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b\": rpc error: code = NotFound desc = could not find container \"6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b\": container with ID starting with 6c9eb6d08dea550f29a84c7890d63a4b3abe92ccccb8fa9e2b0187df4f6a5a5b not found: ID does not exist"
Sep 16 16:29:30.369996 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.369930    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:29:30.373948 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:30.373921    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dc7pkf"]
Sep 16 16:29:31.309836 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:31.309798    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" path="/var/lib/kubelet/pods/472ffc23-a25e-494a-ac62-2e16a3053352/volumes"
Sep 16 16:29:40.874329 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.874277    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:29:40.874903 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.874880    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main"
Sep 16 16:29:40.874948 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.874909    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main"
Sep 16 16:29:40.874987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.874954    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerName="main"
Sep 16 16:29:40.874987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.874964    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerName="main"
Sep 16 16:29:40.875056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.875048    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="472ffc23-a25e-494a-ac62-2e16a3053352" containerName="main"
Sep 16 16:29:40.875091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.875063    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3374c283-91f4-4131-b3f7-a4e7b7615576" containerName="main"
Sep 16 16:29:40.878514 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.878489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.882695 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.882659    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-leave-group-5b69d57a\"/\"openshift-service-ca.crt\""
Sep 16 16:29:40.882842 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.882725    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-leave-group-5b69d57a\"/\"leave-v2-kserve-self-signed-certs\""
Sep 16 16:29:40.882842 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.882802    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-leave-group-5b69d57a\"/\"default-dockercfg-4ksct\""
Sep 16 16:29:40.882991 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.882835    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-leave-group-5b69d57a\"/\"kube-root-ca.crt\""
Sep 16 16:29:40.888142 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.888113    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:29:40.913487 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.913487 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.913715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.913715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.913715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:40.913715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:40.913661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh7cx\" (UniqueName: \"kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014369 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014596 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014596 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014596 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh7cx\" (UniqueName: \"kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014596 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.014596 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.014542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.015038 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.015009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.015038 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.015029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.015238 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.015214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.017301 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.017278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.017632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.017608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.023387 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.023350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh7cx\" (UniqueName: \"kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx\") pod \"leave-v2-kserve-556b4dddf9-sh75g\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") " pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.196087 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.196044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:41.495485 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:41.495457    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:29:41.498188 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:29:41.498137    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2590df33_a503_4177_98f4_d03f52c4e3b9.slice/crio-b0a9dfc6ca05b8dd909943eaad0323999c094ccff8c45f63e6c73d6cb9bb6f90 WatchSource:0}: Error finding container b0a9dfc6ca05b8dd909943eaad0323999c094ccff8c45f63e6c73d6cb9bb6f90: Status 404 returned error can't find the container with id b0a9dfc6ca05b8dd909943eaad0323999c094ccff8c45f63e6c73d6cb9bb6f90
Sep 16 16:29:42.417033 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:42.416995    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" event={"ID":"2590df33-a503-4177-98f4-d03f52c4e3b9","Type":"ContainerStarted","Data":"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"}
Sep 16 16:29:42.417445 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:42.417047    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" event={"ID":"2590df33-a503-4177-98f4-d03f52c4e3b9","Type":"ContainerStarted","Data":"b0a9dfc6ca05b8dd909943eaad0323999c094ccff8c45f63e6c73d6cb9bb6f90"}
Sep 16 16:29:42.435778 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:42.435717    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" podStartSLOduration=2.43570093 podStartE2EDuration="2.43570093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:29:42.433727145 +0000 UTC m=+2379.661185153" watchObservedRunningTime="2026-09-16 16:29:42.43570093 +0000 UTC m=+2379.663158969"
Sep 16 16:29:43.196228 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:43.196187    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:51.196790 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:51.196741    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:51.209429 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:51.209401    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:29:51.475256 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:29:51.475149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:30:03.477403 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:03.477369    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:30:03.480540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:03.480516    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:30:03.483170 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:03.483141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:30:03.486192 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:03.486151    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:30:26.784859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.784782    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:30:26.788465 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.788441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:26.791266 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.791241    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"openshift-service-ca.crt\""
Sep 16 16:30:26.792508 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.792486    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"kube-root-ca.crt\""
Sep 16 16:30:26.792618 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.792529    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"default-dockercfg-29xkz\""
Sep 16 16:30:26.792618 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.792536    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisvc-router-managed-test-llm-e45d1f79-epp-sa-dockercfg-zmqqn\""
Sep 16 16:30:26.792734 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.792497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Sep 16 16:30:26.800546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.800524    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:30:26.916538 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.916501    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nbl7\" (UniqueName: \"kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:26.916704 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:26.916543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.017798 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.017766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nbl7\" (UniqueName: \"kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.017982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.017804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.020344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.020318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.026007 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.025985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nbl7\" (UniqueName: \"kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.100630 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.100540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.376148 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.376119    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:30:27.377829 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:30:27.377799    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68f3d33_03ff_422e_b9d9_1e228cd6135b.slice/crio-e70e0b0dc3b308b3b26c49482b36981f5b6a3d8b04658fbb716ee19c622af52f WatchSource:0}: Error finding container e70e0b0dc3b308b3b26c49482b36981f5b6a3d8b04658fbb716ee19c622af52f: Status 404 returned error can't find the container with id e70e0b0dc3b308b3b26c49482b36981f5b6a3d8b04658fbb716ee19c622af52f
Sep 16 16:30:27.655938 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.655901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" event={"ID":"c68f3d33-03ff-422e-b9d9-1e228cd6135b","Type":"ContainerStarted","Data":"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"}
Sep 16 16:30:27.656132 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.655962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" event={"ID":"c68f3d33-03ff-422e-b9d9-1e228cd6135b","Type":"ContainerStarted","Data":"e70e0b0dc3b308b3b26c49482b36981f5b6a3d8b04658fbb716ee19c622af52f"}
Sep 16 16:30:27.656132 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.656111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:27.679419 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:27.679362    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podStartSLOduration=1.679344876 podStartE2EDuration="1.679344876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:30:27.677106078 +0000 UTC m=+2424.904564079" watchObservedRunningTime="2026-09-16 16:30:27.679344876 +0000 UTC m=+2424.906802883"
Sep 16 16:30:43.062965 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:43.062929    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:30:58.085748 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.085700    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" podUID="2590df33-a503-4177-98f4-d03f52c4e3b9" containerName="main" containerID="cri-o://8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55" gracePeriod=45
Sep 16 16:30:58.335308 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.335280    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:30:58.391213 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391177    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391381 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391233    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh7cx\" (UniqueName: \"kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391381 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391257    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391381 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391294    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391381 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home\") pod \"2590df33-a503-4177-98f4-d03f52c4e3b9\" (UID: \"2590df33-a503-4177-98f4-d03f52c4e3b9\") "
Sep 16 16:30:58.391605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:30:58.391721 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:30:58.391721 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391700    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.391795 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.391768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home" (OuterVolumeSpecName: "home") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:30:58.393692 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.393665    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm" (OuterVolumeSpecName: "dshm") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:30:58.393692 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.393668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:30:58.393854 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.393709    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx" (OuterVolumeSpecName: "kube-api-access-dh7cx") pod "2590df33-a503-4177-98f4-d03f52c4e3b9" (UID: "2590df33-a503-4177-98f4-d03f52c4e3b9"). InnerVolumeSpecName "kube-api-access-dh7cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:30:58.492987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.492951    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.492987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.492983    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh7cx\" (UniqueName: \"kubernetes.io/projected/2590df33-a503-4177-98f4-d03f52c4e3b9-kube-api-access-dh7cx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.492987 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.492994    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2590df33-a503-4177-98f4-d03f52c4e3b9-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.493254 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.493006    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.493254 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.493014    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2590df33-a503-4177-98f4-d03f52c4e3b9-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:30:58.663370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.663277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:30:58.823297 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.823264    2581 generic.go:356] "Generic (PLEG): container finished" podID="2590df33-a503-4177-98f4-d03f52c4e3b9" containerID="8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55" exitCode=0
Sep 16 16:30:58.823461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.823306    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"
Sep 16 16:30:58.823461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.823348    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" event={"ID":"2590df33-a503-4177-98f4-d03f52c4e3b9","Type":"ContainerDied","Data":"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"}
Sep 16 16:30:58.823461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.823404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g" event={"ID":"2590df33-a503-4177-98f4-d03f52c4e3b9","Type":"ContainerDied","Data":"b0a9dfc6ca05b8dd909943eaad0323999c094ccff8c45f63e6c73d6cb9bb6f90"}
Sep 16 16:30:58.823461 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.823432    2581 scope.go:117] "RemoveContainer" containerID="8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"
Sep 16 16:30:58.833575 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.833552    2581 scope.go:117] "RemoveContainer" containerID="8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"
Sep 16 16:30:58.833832 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:30:58.833812    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55\": container with ID starting with 8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55 not found: ID does not exist" containerID="8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"
Sep 16 16:30:58.833899 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.833843    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55"} err="failed to get container status \"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55\": rpc error: code = NotFound desc = could not find container \"8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55\": container with ID starting with 8728f8e90cbe08f1a63700680d7025ddfb69b637105554d489ee2d0120dece55 not found: ID does not exist"
Sep 16 16:30:58.846644 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.846621    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:30:58.850601 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:58.850577    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-leave-group-5b69d57a/leave-v2-kserve-556b4dddf9-sh75g"]
Sep 16 16:30:59.311008 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:30:59.310979    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2590df33-a503-4177-98f4-d03f52c4e3b9" path="/var/lib/kubelet/pods/2590df33-a503-4177-98f4-d03f52c4e3b9/volumes"
Sep 16 16:31:53.688768 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:31:53.688721    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:31:58.664138 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:31:58.664052    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:32:08.663321 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:08.663277    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:32:08.709548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:08.709498    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" containerID="cri-o://10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711" gracePeriod=45
Sep 16 16:32:18.663731 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:18.663687    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:32:18.664132 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:18.663794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:32:28.663469 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:28.663422    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:32:38.663838 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:38.663793    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:32:38.963649 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:38.963623    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:32:39.056752 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.056711    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nbl7\" (UniqueName: \"kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7\") pod \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") "
Sep 16 16:32:39.056951 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.056796    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs\") pod \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\" (UID: \"c68f3d33-03ff-422e-b9d9-1e228cd6135b\") "
Sep 16 16:32:39.059063 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.059037    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7" (OuterVolumeSpecName: "kube-api-access-8nbl7") pod "c68f3d33-03ff-422e-b9d9-1e228cd6135b" (UID: "c68f3d33-03ff-422e-b9d9-1e228cd6135b"). InnerVolumeSpecName "kube-api-access-8nbl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:32:39.059174 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.059058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c68f3d33-03ff-422e-b9d9-1e228cd6135b" (UID: "c68f3d33-03ff-422e-b9d9-1e228cd6135b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:32:39.157940 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.157899    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nbl7\" (UniqueName: \"kubernetes.io/projected/c68f3d33-03ff-422e-b9d9-1e228cd6135b-kube-api-access-8nbl7\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:32:39.157940 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.157937    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c68f3d33-03ff-422e-b9d9-1e228cd6135b-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:32:39.379840 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.379800    2581 generic.go:356] "Generic (PLEG): container finished" podID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerID="10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711" exitCode=0
Sep 16 16:32:39.380018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.379849    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"
Sep 16 16:32:39.380018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.379886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" event={"ID":"c68f3d33-03ff-422e-b9d9-1e228cd6135b","Type":"ContainerDied","Data":"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"}
Sep 16 16:32:39.380018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.379941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm" event={"ID":"c68f3d33-03ff-422e-b9d9-1e228cd6135b","Type":"ContainerDied","Data":"e70e0b0dc3b308b3b26c49482b36981f5b6a3d8b04658fbb716ee19c622af52f"}
Sep 16 16:32:39.380018 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.379965    2581 scope.go:117] "RemoveContainer" containerID="10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"
Sep 16 16:32:39.388963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.388941    2581 scope.go:117] "RemoveContainer" containerID="10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"
Sep 16 16:32:39.389295 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:32:39.389275    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711\": container with ID starting with 10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711 not found: ID does not exist" containerID="10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"
Sep 16 16:32:39.389363 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.389302    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711"} err="failed to get container status \"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711\": rpc error: code = NotFound desc = could not find container \"10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711\": container with ID starting with 10f82fe105c62242569cf973fedf9359cb837f5e76f3a1457ce82af026252711 not found: ID does not exist"
Sep 16 16:32:39.396225 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.396200    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:32:39.400404 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:39.400380    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheh4xqm"]
Sep 16 16:32:41.316123 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:41.316084    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" path="/var/lib/kubelet/pods/c68f3d33-03ff-422e-b9d9-1e228cd6135b/volumes"
Sep 16 16:32:53.733925 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.733884    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:32:53.734500 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734477    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2590df33-a503-4177-98f4-d03f52c4e3b9" containerName="main"
Sep 16 16:32:53.734578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734502    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2590df33-a503-4177-98f4-d03f52c4e3b9" containerName="main"
Sep 16 16:32:53.734578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734513    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main"
Sep 16 16:32:53.734578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734522    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main"
Sep 16 16:32:53.734736 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734628    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c68f3d33-03ff-422e-b9d9-1e228cd6135b" containerName="main"
Sep 16 16:32:53.734736 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.734644    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2590df33-a503-4177-98f4-d03f52c4e3b9" containerName="main"
Sep 16 16:32:53.738247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.738225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.742763 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.742742    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Sep 16 16:32:53.743207 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.743131    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Sep 16 16:32:53.743328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.743202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-ph4nq\""
Sep 16 16:32:53.743328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.743202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Sep 16 16:32:53.743328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.743220    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-q9jsq\""
Sep 16 16:32:53.746328 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.746303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:32:53.889066 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.889028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.889261 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.889094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wmx\" (UniqueName: \"kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.990702 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.989929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.990702 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.990033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wmx\" (UniqueName: \"kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.997032 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.996996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:53.999234 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:53.999211    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wmx\" (UniqueName: \"kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:54.050250 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.050224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:54.327759 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.327731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:32:54.329620 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:32:54.329582    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode946a03b_5173_42ea_a2af_704eb97d1b40.slice/crio-2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682 WatchSource:0}: Error finding container 2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682: Status 404 returned error can't find the container with id 2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682
Sep 16 16:32:54.464520 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.464475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" event={"ID":"e946a03b-5173-42ea-a2af-704eb97d1b40","Type":"ContainerStarted","Data":"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"}
Sep 16 16:32:54.464699 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.464545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" event={"ID":"e946a03b-5173-42ea-a2af-704eb97d1b40","Type":"ContainerStarted","Data":"2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682"}
Sep 16 16:32:54.464699 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.464673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:32:54.481766 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:32:54.481714    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podStartSLOduration=1.4817001570000001 podStartE2EDuration="1.481700157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:32:54.479301884 +0000 UTC m=+2571.706759890" watchObservedRunningTime="2026-09-16 16:32:54.481700157 +0000 UTC m=+2571.709158162"
Sep 16 16:33:25.471720 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:25.471634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:33:38.816286 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.815937    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:33:38.819480 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.819454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.823067 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.823043    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-5jh7s\""
Sep 16 16:33:38.823189 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.823122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Sep 16 16:33:38.828195 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.828149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:33:38.886415 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.886378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.886576 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.886459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvmm6\" (UniqueName: \"kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.987530 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.987483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.987707 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.987560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvmm6\" (UniqueName: \"kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.990379 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.990347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:38.996267 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:38.996242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvmm6\" (UniqueName: \"kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:39.133344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.133311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:39.408714 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.408685    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:33:39.410580 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:33:39.410546    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845a57fb_cf49_4118_97e7_709c6953f703.slice/crio-9595edc8fda9ed4a039921d5bdfa33f7ebb63dddeb4136df49b0a5cd7b0640b1 WatchSource:0}: Error finding container 9595edc8fda9ed4a039921d5bdfa33f7ebb63dddeb4136df49b0a5cd7b0640b1: Status 404 returned error can't find the container with id 9595edc8fda9ed4a039921d5bdfa33f7ebb63dddeb4136df49b0a5cd7b0640b1
Sep 16 16:33:39.715666 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.715571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" event={"ID":"845a57fb-cf49-4118-97e7-709c6953f703","Type":"ContainerStarted","Data":"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"}
Sep 16 16:33:39.715666 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.715627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" event={"ID":"845a57fb-cf49-4118-97e7-709c6953f703","Type":"ContainerStarted","Data":"9595edc8fda9ed4a039921d5bdfa33f7ebb63dddeb4136df49b0a5cd7b0640b1"}
Sep 16 16:33:39.715859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.715684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:33:39.735065 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:33:39.735004    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podStartSLOduration=1.73498321 podStartE2EDuration="1.73498321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:33:39.734097695 +0000 UTC m=+2616.961555704" watchObservedRunningTime="2026-09-16 16:33:39.73498321 +0000 UTC m=+2616.962441219"
Sep 16 16:34:10.723105 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:10.723072    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:34:18.475446 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:18.475404    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:34:20.723544 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:20.723502    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:34:30.723091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:30.723050    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:34:33.502723 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:33.502672    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" containerID="cri-o://a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe" gracePeriod=45
Sep 16 16:34:40.723279 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:40.723241    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:34:40.723751 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:40.723373    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:34:50.723349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:34:50.723303    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:00.723432 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:00.723344    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:03.515972 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.515940    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:35:03.519824 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.519799    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:35:03.522404 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.522386    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:35:03.525940 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.525921    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:35:03.738295 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.738266    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:35:03.875447 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.875355    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs\") pod \"845a57fb-cf49-4118-97e7-709c6953f703\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") "
Sep 16 16:35:03.875623 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.875506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvmm6\" (UniqueName: \"kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6\") pod \"845a57fb-cf49-4118-97e7-709c6953f703\" (UID: \"845a57fb-cf49-4118-97e7-709c6953f703\") "
Sep 16 16:35:03.877941 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.877898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "845a57fb-cf49-4118-97e7-709c6953f703" (UID: "845a57fb-cf49-4118-97e7-709c6953f703"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:35:03.878096 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.877950    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6" (OuterVolumeSpecName: "kube-api-access-kvmm6") pod "845a57fb-cf49-4118-97e7-709c6953f703" (UID: "845a57fb-cf49-4118-97e7-709c6953f703"). InnerVolumeSpecName "kube-api-access-kvmm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:35:03.982330 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.977104    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvmm6\" (UniqueName: \"kubernetes.io/projected/845a57fb-cf49-4118-97e7-709c6953f703-kube-api-access-kvmm6\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:35:03.982330 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:03.977185    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/845a57fb-cf49-4118-97e7-709c6953f703-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:35:04.168020 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.167982    2581 generic.go:356] "Generic (PLEG): container finished" podID="845a57fb-cf49-4118-97e7-709c6953f703" containerID="a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe" exitCode=0
Sep 16 16:35:04.168241 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.168062    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"
Sep 16 16:35:04.168241 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.168074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" event={"ID":"845a57fb-cf49-4118-97e7-709c6953f703","Type":"ContainerDied","Data":"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"}
Sep 16 16:35:04.168241 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.168178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c" event={"ID":"845a57fb-cf49-4118-97e7-709c6953f703","Type":"ContainerDied","Data":"9595edc8fda9ed4a039921d5bdfa33f7ebb63dddeb4136df49b0a5cd7b0640b1"}
Sep 16 16:35:04.168241 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.168191    2581 scope.go:117] "RemoveContainer" containerID="a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"
Sep 16 16:35:04.179057 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.179037    2581 scope.go:117] "RemoveContainer" containerID="a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"
Sep 16 16:35:04.179381 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:35:04.179360    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe\": container with ID starting with a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe not found: ID does not exist" containerID="a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"
Sep 16 16:35:04.179445 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.179388    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe"} err="failed to get container status \"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe\": rpc error: code = NotFound desc = could not find container \"a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe\": container with ID starting with a674752aa17c2c4913fd20dda5b7d5f7c09a427ae35e113862f7846d52558cbe not found: ID does not exist"
Sep 16 16:35:04.192653 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.192627    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:35:04.197449 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:04.197423    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche54r5c"]
Sep 16 16:35:05.311153 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:05.311117    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845a57fb-cf49-4118-97e7-709c6953f703" path="/var/lib/kubelet/pods/845a57fb-cf49-4118-97e7-709c6953f703/volumes"
Sep 16 16:35:11.710957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:11.710908    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:35:15.471407 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:15.471369    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:25.471861 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:25.471823    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:26.737408 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:26.737363    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" containerID="cri-o://af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520" gracePeriod=45
Sep 16 16:35:35.472086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:35.472042    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:35.472660 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:35.472176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:35:45.471471 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:45.471423    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:55.472275 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:55.472236    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:35:56.991812 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:56.991787    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:35:57.157323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.157290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs\") pod \"e946a03b-5173-42ea-a2af-704eb97d1b40\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") "
Sep 16 16:35:57.157488 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.157358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7wmx\" (UniqueName: \"kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx\") pod \"e946a03b-5173-42ea-a2af-704eb97d1b40\" (UID: \"e946a03b-5173-42ea-a2af-704eb97d1b40\") "
Sep 16 16:35:57.159474 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.159442    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx" (OuterVolumeSpecName: "kube-api-access-z7wmx") pod "e946a03b-5173-42ea-a2af-704eb97d1b40" (UID: "e946a03b-5173-42ea-a2af-704eb97d1b40"). InnerVolumeSpecName "kube-api-access-z7wmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:35:57.159561 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.159455    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e946a03b-5173-42ea-a2af-704eb97d1b40" (UID: "e946a03b-5173-42ea-a2af-704eb97d1b40"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:35:57.258399 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.258364    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e946a03b-5173-42ea-a2af-704eb97d1b40-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:35:57.258399 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.258394    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7wmx\" (UniqueName: \"kubernetes.io/projected/e946a03b-5173-42ea-a2af-704eb97d1b40-kube-api-access-z7wmx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:35:57.337553 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:35:57.337522    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode946a03b_5173_42ea_a2af_704eb97d1b40.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode946a03b_5173_42ea_a2af_704eb97d1b40.slice/crio-2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682\": RecentStats: unable to find data in memory cache]"
Sep 16 16:35:57.455344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.455247    2581 generic.go:356] "Generic (PLEG): container finished" podID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerID="af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520" exitCode=0
Sep 16 16:35:57.455344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.455299    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"
Sep 16 16:35:57.455344 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.455333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" event={"ID":"e946a03b-5173-42ea-a2af-704eb97d1b40","Type":"ContainerDied","Data":"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"}
Sep 16 16:35:57.455586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.455389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9" event={"ID":"e946a03b-5173-42ea-a2af-704eb97d1b40","Type":"ContainerDied","Data":"2e6b9bbca6fbc726bb22729fe5124da00c03b3be37d834d0d076a6f19b4b9682"}
Sep 16 16:35:57.455586 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.455419    2581 scope.go:117] "RemoveContainer" containerID="af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"
Sep 16 16:35:57.464352 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.464329    2581 scope.go:117] "RemoveContainer" containerID="af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"
Sep 16 16:35:57.464628 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:35:57.464601    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520\": container with ID starting with af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520 not found: ID does not exist" containerID="af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"
Sep 16 16:35:57.464748 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.464630    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520"} err="failed to get container status \"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520\": rpc error: code = NotFound desc = could not find container \"af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520\": container with ID starting with af1492406a8189f94be1d5218130222e10dda87710a3e37c18a1d4a4bd2b4520 not found: ID does not exist"
Sep 16 16:35:57.472550 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.472524    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:35:57.476448 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:57.476426    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche664k9"]
Sep 16 16:35:59.310414 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:35:59.310383    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" path="/var/lib/kubelet/pods/e946a03b-5173-42ea-a2af-704eb97d1b40/volumes"
Sep 16 16:36:21.530937 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.530902    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:36:21.531566 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531515    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main"
Sep 16 16:36:21.531566 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531537    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main"
Sep 16 16:36:21.531566 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531557    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main"
Sep 16 16:36:21.531566 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531565    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main"
Sep 16 16:36:21.531777 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531650    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="845a57fb-cf49-4118-97e7-709c6953f703" containerName="main"
Sep 16 16:36:21.531777 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.531666    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e946a03b-5173-42ea-a2af-704eb97d1b40" containerName="main"
Sep 16 16:36:21.534802 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.534779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.539332 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.539300    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Sep 16 16:36:21.539454 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.539342    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Sep 16 16:36:21.539454 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.539409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-bgh4r\""
Sep 16 16:36:21.539585 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.539307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Sep 16 16:36:21.539877 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.539716    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-27sbj\""
Sep 16 16:36:21.543915 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.543894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:36:21.564660 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.564632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzjcx\" (UniqueName: \"kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.564809 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.564714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.665090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.665051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.665279 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.665120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjcx\" (UniqueName: \"kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.667595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.667570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.673337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.673309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzjcx\" (UniqueName: \"kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:21.848000 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:21.847904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:22.122584 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:22.122557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:36:22.124341 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:36:22.124307    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebcd84c0_6c87_4f2d_a242_816f35dad737.slice/crio-65f31f3c985e2a390833e482246026c25df8812e7fb789939a5d5f94c9566099 WatchSource:0}: Error finding container 65f31f3c985e2a390833e482246026c25df8812e7fb789939a5d5f94c9566099: Status 404 returned error can't find the container with id 65f31f3c985e2a390833e482246026c25df8812e7fb789939a5d5f94c9566099
Sep 16 16:36:22.591828 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:22.591788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" event={"ID":"ebcd84c0-6c87-4f2d-a242-816f35dad737","Type":"ContainerStarted","Data":"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"}
Sep 16 16:36:22.592425 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:22.591857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" event={"ID":"ebcd84c0-6c87-4f2d-a242-816f35dad737","Type":"ContainerStarted","Data":"65f31f3c985e2a390833e482246026c25df8812e7fb789939a5d5f94c9566099"}
Sep 16 16:36:22.592425 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:22.591892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:36:22.612297 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:22.612248    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podStartSLOduration=1.6122337660000001 podStartE2EDuration="1.612233766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:22.609746894 +0000 UTC m=+2779.837204903" watchObservedRunningTime="2026-09-16 16:36:22.612233766 +0000 UTC m=+2779.839691773"
Sep 16 16:36:53.600042 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:36:53.600009    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:37:26.731056 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.731018    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:37:26.734117 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.734092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.737920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.737898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-7qph6\""
Sep 16 16:37:26.738032 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.737905    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Sep 16 16:37:26.744963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.744934    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:37:26.827954 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.827917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2kbt\" (UniqueName: \"kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.828112 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.827961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.928590 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.928539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2kbt\" (UniqueName: \"kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.928778 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.928595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.931236 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.931213    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:26.937017 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:26.936987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2kbt\" (UniqueName: \"kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:27.046073 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.045967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:27.319934 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.319904    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:37:27.321451 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:37:27.321425    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61bd654_e19a_4651_9b15_fd4ebdf0b1a9.slice/crio-8811533f08636bdbff5afbdf30b6f9c1a8b8d40fe8256f0c06d2f5e43a2b7578 WatchSource:0}: Error finding container 8811533f08636bdbff5afbdf30b6f9c1a8b8d40fe8256f0c06d2f5e43a2b7578: Status 404 returned error can't find the container with id 8811533f08636bdbff5afbdf30b6f9c1a8b8d40fe8256f0c06d2f5e43a2b7578
Sep 16 16:37:27.953270 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.953234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" event={"ID":"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9","Type":"ContainerStarted","Data":"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"}
Sep 16 16:37:27.953657 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.953312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" event={"ID":"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9","Type":"ContainerStarted","Data":"8811533f08636bdbff5afbdf30b6f9c1a8b8d40fe8256f0c06d2f5e43a2b7578"}
Sep 16 16:37:27.953657 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.953385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:37:27.972865 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:27.972797    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podStartSLOduration=1.9727764049999998 podStartE2EDuration="1.972776405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:27.971564332 +0000 UTC m=+2845.199022350" watchObservedRunningTime="2026-09-16 16:37:27.972776405 +0000 UTC m=+2845.200234413"
Sep 16 16:37:58.962219 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:37:58.962131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:38:28.213758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:28.213720    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:38:28.961608 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:28.961563    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:38:38.961661 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:38.961617    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:38:43.245743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:43.245689    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" containerID="cri-o://b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6" gracePeriod=45
Sep 16 16:38:48.961416 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:48.961371    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:38:48.961814 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:48.961501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:38:58.961797 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:38:58.961758    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:39:08.961617 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:08.961576    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:39:13.496876 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.496843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:39:13.542679 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.542585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs\") pod \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") "
Sep 16 16:39:13.542851 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.542707    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2kbt\" (UniqueName: \"kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt\") pod \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\" (UID: \"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9\") "
Sep 16 16:39:13.544892 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.544857    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" (UID: "f61bd654-e19a-4651-9b15-fd4ebdf0b1a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:39:13.545035 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.545017    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt" (OuterVolumeSpecName: "kube-api-access-p2kbt") pod "f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" (UID: "f61bd654-e19a-4651-9b15-fd4ebdf0b1a9"). InnerVolumeSpecName "kube-api-access-p2kbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:39:13.547203 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.547150    2581 generic.go:356] "Generic (PLEG): container finished" podID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerID="b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6" exitCode=0
Sep 16 16:39:13.547337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.547221    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"
Sep 16 16:39:13.547337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.547256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" event={"ID":"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9","Type":"ContainerDied","Data":"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"}
Sep 16 16:39:13.547337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.547308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c" event={"ID":"f61bd654-e19a-4651-9b15-fd4ebdf0b1a9","Type":"ContainerDied","Data":"8811533f08636bdbff5afbdf30b6f9c1a8b8d40fe8256f0c06d2f5e43a2b7578"}
Sep 16 16:39:13.547337 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.547334    2581 scope.go:117] "RemoveContainer" containerID="b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"
Sep 16 16:39:13.567392 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.567367    2581 scope.go:117] "RemoveContainer" containerID="b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"
Sep 16 16:39:13.567724 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:39:13.567701    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6\": container with ID starting with b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6 not found: ID does not exist" containerID="b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"
Sep 16 16:39:13.567808 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.567733    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6"} err="failed to get container status \"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6\": rpc error: code = NotFound desc = could not find container \"b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6\": container with ID starting with b0c3ad7e1bb6e442fcd97757860020a21b4f491586bf49a2ad2b69c9f7ce29f6 not found: ID does not exist"
Sep 16 16:39:13.579112 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.579082    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:39:13.581247 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.581220    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schestp5c"]
Sep 16 16:39:13.644196 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.644149    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2kbt\" (UniqueName: \"kubernetes.io/projected/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-kube-api-access-p2kbt\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:39:13.644196 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:13.644192    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:39:15.310380 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:15.310344    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" path="/var/lib/kubelet/pods/f61bd654-e19a-4651-9b15-fd4ebdf0b1a9/volumes"
Sep 16 16:39:21.744140 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:21.744101    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:39:23.599256 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:23.599215    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:39:33.599646 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:33.599548    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:39:36.763093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:36.763031    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" containerID="cri-o://b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b" gracePeriod=45
Sep 16 16:39:43.600075 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:43.600036    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:39:43.600506 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:43.600139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:39:53.599385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:39:53.599342    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:40:03.552507 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:03.552473    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:40:03.557010 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:03.556983    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:40:03.558273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:03.558252    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:40:03.562578 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:03.562560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:40:03.600039 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:03.600010    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:40:07.017152 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.017122    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:40:07.109916 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.109839    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs\") pod \"ebcd84c0-6c87-4f2d-a242-816f35dad737\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") "
Sep 16 16:40:07.110089 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.109956    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzjcx\" (UniqueName: \"kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx\") pod \"ebcd84c0-6c87-4f2d-a242-816f35dad737\" (UID: \"ebcd84c0-6c87-4f2d-a242-816f35dad737\") "
Sep 16 16:40:07.111993 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.111958    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebcd84c0-6c87-4f2d-a242-816f35dad737" (UID: "ebcd84c0-6c87-4f2d-a242-816f35dad737"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:40:07.112209 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.112150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx" (OuterVolumeSpecName: "kube-api-access-fzjcx") pod "ebcd84c0-6c87-4f2d-a242-816f35dad737" (UID: "ebcd84c0-6c87-4f2d-a242-816f35dad737"). InnerVolumeSpecName "kube-api-access-fzjcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:40:07.211273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.211223    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzjcx\" (UniqueName: \"kubernetes.io/projected/ebcd84c0-6c87-4f2d-a242-816f35dad737-kube-api-access-fzjcx\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:40:07.211273 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.211267    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd84c0-6c87-4f2d-a242-816f35dad737-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:40:07.854648 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.854615    2581 generic.go:356] "Generic (PLEG): container finished" podID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerID="b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b" exitCode=0
Sep 16 16:40:07.854829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.854662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"
Sep 16 16:40:07.854829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.854700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" event={"ID":"ebcd84c0-6c87-4f2d-a242-816f35dad737","Type":"ContainerDied","Data":"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"}
Sep 16 16:40:07.854829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.854757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh" event={"ID":"ebcd84c0-6c87-4f2d-a242-816f35dad737","Type":"ContainerDied","Data":"65f31f3c985e2a390833e482246026c25df8812e7fb789939a5d5f94c9566099"}
Sep 16 16:40:07.854829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.854782    2581 scope.go:117] "RemoveContainer" containerID="b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"
Sep 16 16:40:07.864150 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.864133    2581 scope.go:117] "RemoveContainer" containerID="b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"
Sep 16 16:40:07.864598 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:40:07.864576    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b\": container with ID starting with b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b not found: ID does not exist" containerID="b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"
Sep 16 16:40:07.864689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.864603    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b"} err="failed to get container status \"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b\": rpc error: code = NotFound desc = could not find container \"b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b\": container with ID starting with b688a3c5b38c159df027b7088dab9789141d23cd45d82fded638ba785a00487b not found: ID does not exist"
Sep 16 16:40:07.871605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.871581    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:40:07.875453 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:07.875433    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schesglfh"]
Sep 16 16:40:09.310006 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:09.309971    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" path="/var/lib/kubelet/pods/ebcd84c0-6c87-4f2d-a242-816f35dad737/volumes"
Sep 16 16:40:27.677251 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677210    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:40:27.677756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677691    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main"
Sep 16 16:40:27.677756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677706    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main"
Sep 16 16:40:27.677756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677726    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main"
Sep 16 16:40:27.677756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677732    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main"
Sep 16 16:40:27.677907 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677792    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61bd654-e19a-4651-9b15-fd4ebdf0b1a9" containerName="main"
Sep 16 16:40:27.677907 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.677800    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebcd84c0-6c87-4f2d-a242-816f35dad737" containerName="main"
Sep 16 16:40:27.680874 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.680853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.684752 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.684728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"kube-root-ca.crt\""
Sep 16 16:40:27.684913 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.684728    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"openshift-service-ca.crt\""
Sep 16 16:40:27.684913 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.684728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"default-dockercfg-t94p6\""
Sep 16 16:40:27.685046 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.684728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v1-kserve-self-signed-certs\""
Sep 16 16:40:27.691557 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.691533    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:40:27.778876 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.778835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.778876 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.778879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.779090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.778906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7r56\" (UniqueName: \"kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.779090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.778986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.779090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.779046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.779090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.779071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880267 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880267 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7r56\" (UniqueName: \"kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880796 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880796 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.880982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.880835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.882551 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.882526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.882826 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.882807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.888152 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.888123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7r56\" (UniqueName: \"kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56\") pod \"owner-v1-kserve-75579ff5bc-tw4pd\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:27.993676 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:27.993593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:28.270005 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:28.269970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:40:28.967624 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:28.967584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" event={"ID":"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1","Type":"ContainerStarted","Data":"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"}
Sep 16 16:40:28.968003 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:28.967650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" event={"ID":"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1","Type":"ContainerStarted","Data":"38f4d367f063b71052579fd065be9f4b57b8b8ef8e250e2b46c04128216ff0f6"}
Sep 16 16:40:28.986398 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:28.986344    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" podStartSLOduration=1.986329507 podStartE2EDuration="1.986329507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:28.984517016 +0000 UTC m=+3026.211975021" watchObservedRunningTime="2026-09-16 16:40:28.986329507 +0000 UTC m=+3026.213787513"
Sep 16 16:40:28.994667 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:28.994643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:37.994238 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:37.994198    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:38.006590 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:38.006558    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:38.027233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:38.027204    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:40:52.873981 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.873942    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:40:52.878110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.878087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.880768 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.880745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v2-kserve-self-signed-certs\""
Sep 16 16:40:52.886069 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886037    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:40:52.886361 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.886453 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvp7l\" (UniqueName: \"kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.886453 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.886572 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.886627 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.886679 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.886633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987544 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvp7l\" (UniqueName: \"kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987745 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.987988 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.988034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.987999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.988034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.988024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.989853 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.989832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.990260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.990240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:52.995576 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:52.995556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvp7l\" (UniqueName: \"kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l\") pod \"owner-v2-kserve-5cf8959f95-6pckr\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:53.191881 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:53.191847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:40:53.469525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:53.469215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:40:54.104349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:54.104308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" event={"ID":"f1137810-3d68-40bc-a010-afeaab412261","Type":"ContainerStarted","Data":"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"}
Sep 16 16:40:54.104756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:54.104361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" event={"ID":"f1137810-3d68-40bc-a010-afeaab412261","Type":"ContainerStarted","Data":"3851340cce7df2543e04fa1db8c89e7afa873689a4194aba66ce497769a602c5"}
Sep 16 16:40:54.122109 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:54.122061    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" podStartSLOduration=2.122047371 podStartE2EDuration="2.122047371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:54.12019523 +0000 UTC m=+3051.347653236" watchObservedRunningTime="2026-09-16 16:40:54.122047371 +0000 UTC m=+3051.349505377"
Sep 16 16:40:54.192180 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:40:54.192133    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:41:03.192869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:03.192774    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:41:03.205549 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:03.205514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:41:04.167397 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:04.167361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:41:43.346807 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:43.346765    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:41:58.373758 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.373695    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" podUID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" containerName="main" containerID="cri-o://47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce" gracePeriod=45
Sep 16 16:41:58.632630 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.632602    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:41:58.636986 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.636963    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7r56\" (UniqueName: \"kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.636996    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637053    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637109    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637366 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637130    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache\") pod \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\" (UID: \"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1\") "
Sep 16 16:41:58.637366 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home" (OuterVolumeSpecName: "home") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:41:58.637465 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637401    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:41:58.637516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637456    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:58.637516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.637478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:41:58.639375 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.639353    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56" (OuterVolumeSpecName: "kube-api-access-j7r56") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "kube-api-access-j7r56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:41:58.639499 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.639389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm" (OuterVolumeSpecName: "dshm") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:41:58.639499 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.639390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" (UID: "0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:41:58.738511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.738475    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7r56\" (UniqueName: \"kubernetes.io/projected/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-kube-api-access-j7r56\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:58.738511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.738506    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:58.738511 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.738516    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:58.738761 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.738525    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:58.738761 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:58.738535    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:41:59.457541 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.457485    2581 generic.go:356] "Generic (PLEG): container finished" podID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" containerID="47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce" exitCode=0
Sep 16 16:41:59.457541 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.457538    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"
Sep 16 16:41:59.458059 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.457566    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" event={"ID":"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1","Type":"ContainerDied","Data":"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"}
Sep 16 16:41:59.458059 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.457630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd" event={"ID":"0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1","Type":"ContainerDied","Data":"38f4d367f063b71052579fd065be9f4b57b8b8ef8e250e2b46c04128216ff0f6"}
Sep 16 16:41:59.458059 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.457651    2581 scope.go:117] "RemoveContainer" containerID="47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"
Sep 16 16:41:59.468177 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.468131    2581 scope.go:117] "RemoveContainer" containerID="47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"
Sep 16 16:41:59.468448 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:41:59.468425    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce\": container with ID starting with 47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce not found: ID does not exist" containerID="47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"
Sep 16 16:41:59.468524 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.468460    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce"} err="failed to get container status \"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce\": rpc error: code = NotFound desc = could not find container \"47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce\": container with ID starting with 47d73d9a12c6fea9c351beae2533c66068ac755e4288006f54f61edd9657a6ce not found: ID does not exist"
Sep 16 16:41:59.475659 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.475637    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:41:59.481272 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:41:59.481247    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v1-kserve-75579ff5bc-tw4pd"]
Sep 16 16:42:01.310090 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:01.310055    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" path="/var/lib/kubelet/pods/0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1/volumes"
Sep 16 16:42:06.726948 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.726909    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:42:06.727431 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.727411    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" containerName="main"
Sep 16 16:42:06.727497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.727435    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" containerName="main"
Sep 16 16:42:06.727571 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.727558    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fdc2866-912f-4f25-a4e9-3ee36a0d2dc1" containerName="main"
Sep 16 16:42:06.732428 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.732399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.735595 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.735569    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-gw-ca-root-cert\""
Sep 16 16:42:06.735730 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.735694    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Sep 16 16:42:06.736348 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.736330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Sep 16 16:42:06.736886 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.736865    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"router-gateway-1-openshift-default-dockercfg-7ws9n\""
Sep 16 16:42:06.750313 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.750286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:42:06.807306 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdxgn\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.807632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.807568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908535 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908764 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908764 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdxgn\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908764 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908764 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.908764 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909087 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909087 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.908955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909087 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.909011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909087 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.909033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909472 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.909252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909472 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.909335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.909472 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.909437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.911323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.911301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.911439 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.911324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.929691 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.929670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:06.929886 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:06.929868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdxgn\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-dpgck\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:07.046060 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:07.045960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:07.326512 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:07.325092    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:42:07.332322 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:07.332302    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:42:07.502472 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:07.502428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerStarted","Data":"16d8fd94568a96e20c2c57ae8b5ee49c3b54bd52dc9eef44b9176135aeb34808"}
Sep 16 16:42:09.960902 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:09.960865    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:42:10.420827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:10.420784    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:10.420934 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:10.420863    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:10.420934 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:10.420893    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:10.522299 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:10.522207    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerStarted","Data":"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"}
Sep 16 16:42:10.548852 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:10.548797    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podStartSLOduration=1.460658301 podStartE2EDuration="4.548781975s" totalImagesPullingTime="3.088123674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:42:10.546389778 +0000 UTC m=+3127.773847784" watchObservedRunningTime="2026-09-16 16:42:10.548781975 +0000 UTC m=+3127.776239981"
Sep 16 16:42:11.046929 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:11.046885    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:11.052542 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:11.052510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:11.527933 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:11.527900    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:11.528770 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:11.528750    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:24.982765 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:24.982696    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" podUID="f1137810-3d68-40bc-a010-afeaab412261" containerName="main" containerID="cri-o://4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe" gracePeriod=45
Sep 16 16:42:25.230038 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.230013    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:42:25.379213 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379072    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379213 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379213 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379213 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379184    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379223    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvp7l\" (UniqueName: \"kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l\") pod \"f1137810-3d68-40bc-a010-afeaab412261\" (UID: \"f1137810-3d68-40bc-a010-afeaab412261\") "
Sep 16 16:42:25.379547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:25.379547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379479    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home" (OuterVolumeSpecName: "home") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:25.379547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379523    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.379805 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.379625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:25.381383 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.381351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm" (OuterVolumeSpecName: "dshm") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:25.381513 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.381394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:25.381513 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.381439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l" (OuterVolumeSpecName: "kube-api-access-bvp7l") pod "f1137810-3d68-40bc-a010-afeaab412261" (UID: "f1137810-3d68-40bc-a010-afeaab412261"). InnerVolumeSpecName "kube-api-access-bvp7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:25.480790 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.480758    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.480790 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.480787    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1137810-3d68-40bc-a010-afeaab412261-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.480790 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.480796    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.481037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.480805    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1137810-3d68-40bc-a010-afeaab412261-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.481037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.480813    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvp7l\" (UniqueName: \"kubernetes.io/projected/f1137810-3d68-40bc-a010-afeaab412261-kube-api-access-bvp7l\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:42:25.607495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.607454    2581 generic.go:356] "Generic (PLEG): container finished" podID="f1137810-3d68-40bc-a010-afeaab412261" containerID="4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe" exitCode=0
Sep 16 16:42:25.607664 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.607513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" event={"ID":"f1137810-3d68-40bc-a010-afeaab412261","Type":"ContainerDied","Data":"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"}
Sep 16 16:42:25.607664 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.607518    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"
Sep 16 16:42:25.607664 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.607560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" event={"ID":"f1137810-3d68-40bc-a010-afeaab412261","Type":"ContainerDied","Data":"3851340cce7df2543e04fa1db8c89e7afa873689a4194aba66ce497769a602c5"}
Sep 16 16:42:25.607664 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.607581    2581 scope.go:117] "RemoveContainer" containerID="4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"
Sep 16 16:42:25.616957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.616937    2581 scope.go:117] "RemoveContainer" containerID="4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"
Sep 16 16:42:25.617290 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:42:25.617269    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe\": container with ID starting with 4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe not found: ID does not exist" containerID="4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"
Sep 16 16:42:25.617358 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.617297    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe"} err="failed to get container status \"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe\": rpc error: code = NotFound desc = could not find container \"4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe\": container with ID starting with 4eaca6ec1236b473adef41eae6384ac20f75d4f5fcee830a4c6f1cab1c5978fe not found: ID does not exist"
Sep 16 16:42:25.629921 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.629894    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:42:25.637011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:25.636985    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr"]
Sep 16 16:42:26.157272 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:26.157232    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-6pckr" podUID="f1137810-3d68-40bc-a010-afeaab412261" containerName="main" probeResult="failure" output="Get \"https://10.133.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 16 16:42:27.311632 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:27.311599    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1137810-3d68-40bc-a010-afeaab412261" path="/var/lib/kubelet/pods/f1137810-3d68-40bc-a010-afeaab412261/volumes"
Sep 16 16:42:29.458644 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.458608    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:42:29.459234 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.459002    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1137810-3d68-40bc-a010-afeaab412261" containerName="main"
Sep 16 16:42:29.459234 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.459017    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1137810-3d68-40bc-a010-afeaab412261" containerName="main"
Sep 16 16:42:29.459234 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.459111    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1137810-3d68-40bc-a010-afeaab412261" containerName="main"
Sep 16 16:42:29.462314 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.462291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.467169 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.467122    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Sep 16 16:42:29.467169 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.467142    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"default-dockercfg-8t5mm\""
Sep 16 16:42:29.472234 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.472212    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:42:29.614928 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.614888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76sbp\" (UniqueName: \"kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.615078 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.614934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.615078 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.614993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.615078 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.615047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.615223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.615113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.615268 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.615235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76sbp\" (UniqueName: \"kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716729 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716729 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716815 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.716815 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.716773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.717091 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.717056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.717260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.717119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.717260 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.717141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.718833 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.718813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.719001 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.718981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.727413 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.727389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76sbp\" (UniqueName: \"kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:29.776369 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:29.776346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:30.056636 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:42:30.056607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b263a1_f42c_4da3_bcc9_c14a136ca46c.slice/crio-2af6040cca9b3f36fdd62695609932c59b69f7e6f36419be69a17152242a7a0d WatchSource:0}: Error finding container 2af6040cca9b3f36fdd62695609932c59b69f7e6f36419be69a17152242a7a0d: Status 404 returned error can't find the container with id 2af6040cca9b3f36fdd62695609932c59b69f7e6f36419be69a17152242a7a0d
Sep 16 16:42:30.056829 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:30.056800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:42:30.638242 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:30.638202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" event={"ID":"16b263a1-f42c-4da3-bcc9-c14a136ca46c","Type":"ContainerStarted","Data":"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"}
Sep 16 16:42:30.638688 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:30.638266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" event={"ID":"16b263a1-f42c-4da3-bcc9-c14a136ca46c","Type":"ContainerStarted","Data":"2af6040cca9b3f36fdd62695609932c59b69f7e6f36419be69a17152242a7a0d"}
Sep 16 16:42:30.660327 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:30.660251    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" podStartSLOduration=1.660227179 podStartE2EDuration="1.660227179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:42:30.657865152 +0000 UTC m=+3147.885323160" watchObservedRunningTime="2026-09-16 16:42:30.660227179 +0000 UTC m=+3147.887685186"
Sep 16 16:42:30.776887 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:30.776847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:39.718532 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.718472    2581 generic.go:356] "Generic (PLEG): container finished" podID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerID="8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864" exitCode=0
Sep 16 16:42:39.718963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.718550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerDied","Data":"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"}
Sep 16 16:42:39.718963 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.718936    2581 scope.go:117] "RemoveContainer" containerID="8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"
Sep 16 16:42:39.719848 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.719817    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:39.719964 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.719889    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:39.719964 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.719920    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:42:39.776636 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.776602    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:39.791780 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:39.791747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:40.726454 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:40.726411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerStarted","Data":"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"}
Sep 16 16:42:40.737824 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:40.737788    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:42:41.046403 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:41.046306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:41.047933 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:41.047897    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 16 16:42:41.528034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:41.527999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:42.047301 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:42.047249    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 16 16:42:43.046882 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:43.046829    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 16 16:42:44.047335 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:44.047291    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 16 16:42:45.047504 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:45.047452    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 16 16:42:46.050602 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:46.050564    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:42:46.051675 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:42:46.051656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:43:46.413637 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.413601    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:43:46.422543 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:43:46.422501    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 16 16:43:46.422543 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:43:46.422543 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:43:46.422543 ip-10-0-137-111 kubenswrapper[2581]:  > containerID="c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd" cmd=["/bin/sleep","15"]
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.422556    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" message="" err=<
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:  >
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:43:46.422580    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]:  > pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main" containerID="cri-o://c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"
Sep 16 16:43:46.422848 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.422613    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main" containerID="cri-o://c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd" gracePeriod=60
Sep 16 16:43:46.771200 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.771173    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:43:46.803327 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803390    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803412    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76sbp\" (UniqueName: \"kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803430    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803497 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803455    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home\") pod \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\" (UID: \"16b263a1-f42c-4da3-bcc9-c14a136ca46c\") "
Sep 16 16:43:46.803795 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:46.803795 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803723    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache" (OuterVolumeSpecName: "model-cache") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:46.803936 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.803872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home" (OuterVolumeSpecName: "home") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:46.805900 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.805872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp" (OuterVolumeSpecName: "kube-api-access-76sbp") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "kube-api-access-76sbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:43:46.806009 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.805900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm" (OuterVolumeSpecName: "dshm") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:43:46.806009 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.805907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16b263a1-f42c-4da3-bcc9-c14a136ca46c" (UID: "16b263a1-f42c-4da3-bcc9-c14a136ca46c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:46.905134 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905096    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:46.905134 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905127    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76sbp\" (UniqueName: \"kubernetes.io/projected/16b263a1-f42c-4da3-bcc9-c14a136ca46c-kube-api-access-76sbp\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:46.905134 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905139    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-model-cache\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:46.905475 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905149    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-home\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:46.905475 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905170    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-dshm\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:46.905475 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:46.905178    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b263a1-f42c-4da3-bcc9-c14a136ca46c-tmp-dir\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:43:47.086442 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.086348    2581 generic.go:356] "Generic (PLEG): container finished" podID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerID="c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd" exitCode=0
Sep 16 16:43:47.086442 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.086398    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"
Sep 16 16:43:47.086628 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.086437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" event={"ID":"16b263a1-f42c-4da3-bcc9-c14a136ca46c","Type":"ContainerDied","Data":"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"}
Sep 16 16:43:47.086628 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.086495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" event={"ID":"16b263a1-f42c-4da3-bcc9-c14a136ca46c","Type":"ContainerDied","Data":"2af6040cca9b3f36fdd62695609932c59b69f7e6f36419be69a17152242a7a0d"}
Sep 16 16:43:47.086628 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.086518    2581 scope.go:117] "RemoveContainer" containerID="c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"
Sep 16 16:43:47.096537 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.096517    2581 scope.go:117] "RemoveContainer" containerID="c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"
Sep 16 16:43:47.096818 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:43:47.096798    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd\": container with ID starting with c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd not found: ID does not exist" containerID="c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"
Sep 16 16:43:47.096880 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.096826    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd"} err="failed to get container status \"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd\": rpc error: code = NotFound desc = could not find container \"c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd\": container with ID starting with c920388521538b067b79d8b6bdc6c62d5811b880610edcb56b5d11568b06b9bd not found: ID does not exist"
Sep 16 16:43:47.132516 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.132484    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:43:47.140786 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.140755    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw"]
Sep 16 16:43:47.316980 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.316947    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" path="/var/lib/kubelet/pods/16b263a1-f42c-4da3-bcc9-c14a136ca46c/volumes"
Sep 16 16:43:47.727212 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:47.727134    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997nkqdw" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/health\": context deadline exceeded"
Sep 16 16:43:56.425279 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:56.425197    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:43:56.425718 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:43:56.425416    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" containerID="cri-o://80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770" gracePeriod=30
Sep 16 16:44:01.053697 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.053651    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:44:01.679614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.679584    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:44:01.744710 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744673    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdxgn\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.744884 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744741    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.744884 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.744884 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744810    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.744884 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744857    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.745044 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744893    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.745044 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.745044 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.744960    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.745044 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket\") pod \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\" (UID: \"c6d30e9b-590d-4b03-9c99-4894b80cc0b7\") "
Sep 16 16:44:01.745533 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745505    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:01.745724 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745640    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:01.745836 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:01.745941 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745832    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:44:01.746033 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.745843    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:01.747374 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.747299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:44:01.747466 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.747422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:44:01.747466 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.747446    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn" (OuterVolumeSpecName: "kube-api-access-zdxgn") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "kube-api-access-zdxgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:44:01.747878 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.747853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6d30e9b-590d-4b03-9c99-4894b80cc0b7" (UID: "c6d30e9b-590d-4b03-9c99-4894b80cc0b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:01.846548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846501    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846544    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-token\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846553    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istiod-ca-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846561    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-envoy\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846569    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-credential-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846578    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdxgn\" (UniqueName: \"kubernetes.io/projected/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-kube-api-access-zdxgn\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846587    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-podinfo\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846595    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-istio-data\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:01.846806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:01.846603    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6d30e9b-590d-4b03-9c99-4894b80cc0b7-workload-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:44:02.174380 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.174347    2581 generic.go:356] "Generic (PLEG): container finished" podID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerID="80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770" exitCode=0
Sep 16 16:44:02.174859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.174409    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"
Sep 16 16:44:02.174859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.174432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerDied","Data":"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"}
Sep 16 16:44:02.174859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.174485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck" event={"ID":"c6d30e9b-590d-4b03-9c99-4894b80cc0b7","Type":"ContainerDied","Data":"16d8fd94568a96e20c2c57ae8b5ee49c3b54bd52dc9eef44b9176135aeb34808"}
Sep 16 16:44:02.174859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.174508    2581 scope.go:117] "RemoveContainer" containerID="80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"
Sep 16 16:44:02.185142 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.185115    2581 scope.go:117] "RemoveContainer" containerID="8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"
Sep 16 16:44:02.194581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.194559    2581 scope.go:117] "RemoveContainer" containerID="80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"
Sep 16 16:44:02.194879 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:44:02.194859    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770\": container with ID starting with 80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770 not found: ID does not exist" containerID="80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"
Sep 16 16:44:02.194930 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.194891    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770"} err="failed to get container status \"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770\": rpc error: code = NotFound desc = could not find container \"80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770\": container with ID starting with 80db6ee2a5898079654126a15f50e84839e1f1ff73de61cdf31a86ffe1d31770 not found: ID does not exist"
Sep 16 16:44:02.194930 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.194911    2581 scope.go:117] "RemoveContainer" containerID="8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"
Sep 16 16:44:02.195322 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:44:02.195293    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864\": container with ID starting with 8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864 not found: ID does not exist" containerID="8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"
Sep 16 16:44:02.195401 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.195329    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864"} err="failed to get container status \"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864\": rpc error: code = NotFound desc = could not find container \"8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864\": container with ID starting with 8233402c5f48d30b9b47b1ca2986be52e3471c6afcd89acefc89c0e57f855864 not found: ID does not exist"
Sep 16 16:44:02.200404 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.200376    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:44:02.203747 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:02.203719    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-dpgck"]
Sep 16 16:44:03.310407 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:03.310376    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" path="/var/lib/kubelet/pods/c6d30e9b-590d-4b03-9c99-4894b80cc0b7/volumes"
Sep 16 16:44:21.747369 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747332    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747677    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747689    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747710    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747716    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747723    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747729    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747784    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:44:21.747908 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.747791    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b263a1-f42c-4da3-bcc9-c14a136ca46c" containerName="main"
Sep 16 16:44:21.750852 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.750826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:21.754677 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.754649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Sep 16 16:44:21.754816 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.754707    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-fxkfl\""
Sep 16 16:44:21.754886 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.754826    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"openshift-service-ca.crt\""
Sep 16 16:44:21.754985 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.754962    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"kube-root-ca.crt\""
Sep 16 16:44:21.755043 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.754967    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"default-dockercfg-zqm6x\""
Sep 16 16:44:21.761795 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.761774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:44:21.924305 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.924267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w577s\" (UniqueName: \"kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:21.924305 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:21.924315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.025666 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.025571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w577s\" (UniqueName: \"kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.025666 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.025626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.028084 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.028064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.034130 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.034101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w577s\" (UniqueName: \"kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.064797 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.064768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:22.342375 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:22.342338    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:44:22.344211 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:44:22.344152    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf03306f_4961_4a55_bc3d_faa27ed0cbae.slice/crio-8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec WatchSource:0}: Error finding container 8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec: Status 404 returned error can't find the container with id 8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec
Sep 16 16:44:23.290805 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:23.290768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" event={"ID":"cf03306f-4961-4a55-bc3d-faa27ed0cbae","Type":"ContainerStarted","Data":"91c7101e84ad7108fd13218962c0515c5d1c920ced47da098a40753d7e20487f"}
Sep 16 16:44:23.291331 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:23.290829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" event={"ID":"cf03306f-4961-4a55-bc3d-faa27ed0cbae","Type":"ContainerStarted","Data":"8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec"}
Sep 16 16:44:23.291331 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:23.290901    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:44:23.307482 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:23.307433    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podStartSLOduration=2.307418685 podStartE2EDuration="2.307418685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:44:23.306280241 +0000 UTC m=+3260.533738247" watchObservedRunningTime="2026-09-16 16:44:23.307418685 +0000 UTC m=+3260.534876718"
Sep 16 16:44:54.297746 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:44:54.297714    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:45:03.592820 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:03.592789    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:45:03.598382 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:03.598359    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:45:03.599575 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:03.599552    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:45:03.604772 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:03.604757    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:45:43.695189 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:43.695135    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:45:44.298232 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:44.298187    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:45:54.298346 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:54.298306    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:45:58.716080 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:45:58.716031    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" containerID="cri-o://91c7101e84ad7108fd13218962c0515c5d1c920ced47da098a40753d7e20487f" gracePeriod=45
Sep 16 16:46:04.298132 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:04.298093    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:46:04.298547 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:04.298216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:46:14.297970 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:14.297922    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:46:24.298173 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:24.298114    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:46:28.949892 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:28.949863    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerID="91c7101e84ad7108fd13218962c0515c5d1c920ced47da098a40753d7e20487f" exitCode=0
Sep 16 16:46:28.950253 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:28.949932    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" event={"ID":"cf03306f-4961-4a55-bc3d-faa27ed0cbae","Type":"ContainerDied","Data":"91c7101e84ad7108fd13218962c0515c5d1c920ced47da098a40753d7e20487f"}
Sep 16 16:46:28.950253 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:28.949982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp" event={"ID":"cf03306f-4961-4a55-bc3d-faa27ed0cbae","Type":"ContainerDied","Data":"8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec"}
Sep 16 16:46:28.950253 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:28.949998    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8033f021a4bf9dfa7618f0f01e3d8c469ca6939b235ecffe21c13294e119daec"
Sep 16 16:46:28.962125 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:28.962105    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:46:29.106746 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.106642    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs\") pod \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") "
Sep 16 16:46:29.106746 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.106731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w577s\" (UniqueName: \"kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s\") pod \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\" (UID: \"cf03306f-4961-4a55-bc3d-faa27ed0cbae\") "
Sep 16 16:46:29.108920 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.108886    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf03306f-4961-4a55-bc3d-faa27ed0cbae" (UID: "cf03306f-4961-4a55-bc3d-faa27ed0cbae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:46:29.109043 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.108948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s" (OuterVolumeSpecName: "kube-api-access-w577s") pod "cf03306f-4961-4a55-bc3d-faa27ed0cbae" (UID: "cf03306f-4961-4a55-bc3d-faa27ed0cbae"). InnerVolumeSpecName "kube-api-access-w577s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:46:29.207739 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.207698    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf03306f-4961-4a55-bc3d-faa27ed0cbae-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:46:29.207739 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.207733    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w577s\" (UniqueName: \"kubernetes.io/projected/cf03306f-4961-4a55-bc3d-faa27ed0cbae-kube-api-access-w577s\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:46:29.955507 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.955473    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"
Sep 16 16:46:29.977994 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.977963    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:46:29.982760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:29.982733    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche9d6dp"]
Sep 16 16:46:31.310370 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:31.310327    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" path="/var/lib/kubelet/pods/cf03306f-4961-4a55-bc3d-faa27ed0cbae/volumes"
Sep 16 16:46:46.436701 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.436661    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:46:46.437356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.437139    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main"
Sep 16 16:46:46.437356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.437180    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main"
Sep 16 16:46:46.437356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.437294    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d30e9b-590d-4b03-9c99-4894b80cc0b7" containerName="istio-proxy"
Sep 16 16:46:46.437356 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.437308    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf03306f-4961-4a55-bc3d-faa27ed0cbae" containerName="main"
Sep 16 16:46:46.440458 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.440436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.444309 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.444278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"openshift-service-ca.crt\""
Sep 16 16:46:46.444444 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.444326    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"default-dockercfg-p5skd\""
Sep 16 16:46:46.444444 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.444380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"kube-root-ca.crt\""
Sep 16 16:46:46.444444 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.444334    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-kserve-self-signed-certs\""
Sep 16 16:46:46.444625 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.444496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-epp-sa-dockercfg-kf4jb\""
Sep 16 16:46:46.451114 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.451093    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:46:46.554977 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.554936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.555178 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.554978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgjp2\" (UniqueName: \"kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.656233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.656190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.656233 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.656234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgjp2\" (UniqueName: \"kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.658772 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.658743    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.664866 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.664836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgjp2\" (UniqueName: \"kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:46.754982 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:46.754900    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:47.033868 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:47.033839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:46:47.035599 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:46:47.035568    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc00d9e9_c81d_4af2_89bb_8ee43edc4e17.slice/crio-181cce837f569a1538f7de0840d34e377803a01ae34b9286d719ee5f1c58b5a0 WatchSource:0}: Error finding container 181cce837f569a1538f7de0840d34e377803a01ae34b9286d719ee5f1c58b5a0: Status 404 returned error can't find the container with id 181cce837f569a1538f7de0840d34e377803a01ae34b9286d719ee5f1c58b5a0
Sep 16 16:46:47.050788 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:47.050750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" event={"ID":"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17","Type":"ContainerStarted","Data":"181cce837f569a1538f7de0840d34e377803a01ae34b9286d719ee5f1c58b5a0"}
Sep 16 16:46:48.058209 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:48.058145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" event={"ID":"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17","Type":"ContainerStarted","Data":"d3bf7e375c3bc4ebbd3099bb9d42de58903c16eacc99da6f3bfa50961f0fd67b"}
Sep 16 16:46:48.058627 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:48.058245    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:46:48.076039 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:48.075982    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" podStartSLOduration=2.075966298 podStartE2EDuration="2.075966298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:48.073740432 +0000 UTC m=+3405.301198441" watchObservedRunningTime="2026-09-16 16:46:48.075966298 +0000 UTC m=+3405.303424304"
Sep 16 16:46:48.168949 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:48.168901    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:48.169140 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:48.169005    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:46:48.668983005 +0000 UTC m=+3405.896441003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:48.190605 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:48.190569    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:46:48.673647 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:48.673608    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:48.673817 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:48.673686    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:46:49.673668755 +0000 UTC m=+3406.901126759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:49.063524 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:46:49.063432    2581 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" secret="" err="secret \"e2e-del-blocked-svc-epp-sa-dockercfg-kf4jb\" not found"
Sep 16 16:46:49.683246 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:49.683214    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:49.683420 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:49.683282    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:46:51.683267288 +0000 UTC m=+3408.910725272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:51.701839 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:51.701799    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:51.702280 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:51.701882    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:46:55.701866963 +0000 UTC m=+3412.929324947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:55.741703 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:55.741616    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:46:55.741703 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:46:55.741694    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:47:03.741679741 +0000 UTC m=+3420.969137724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:47:03.815454 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:47:03.815418    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:47:03.815843 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:47:03.815504    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:47:19.815487989 +0000 UTC m=+3437.042945974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:47:05.087540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:05.087471    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main" containerID="cri-o://d3bf7e375c3bc4ebbd3099bb9d42de58903c16eacc99da6f3bfa50961f0fd67b" gracePeriod=45
Sep 16 16:47:19.858432 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:47:19.858391    2581 secret.go:189] Couldn't get secret e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-self-signed-certs: secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:47:19.858955 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:47:19.858475    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs podName:dc00d9e9-c81d-4af2-89bb-8ee43edc4e17 nodeName:}" failed. No retries permitted until 2026-09-16 16:47:51.858461482 +0000 UTC m=+3469.085919466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs") pod "e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17") : secret "e2e-del-blocked-svc-kserve-self-signed-certs" not found
Sep 16 16:47:20.072171 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.072126    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:47:20.933476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.933426    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:47:20.937230 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.937206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:20.939976 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.939950    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Sep 16 16:47:20.941071 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.941048    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Sep 16 16:47:20.941237 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.941047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-wwq9r\""
Sep 16 16:47:20.941323 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.941283    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Sep 16 16:47:20.941388 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.941172    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-2lbdb\""
Sep 16 16:47:20.950137 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.950108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:47:20.967560 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.967527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6k2z\" (UniqueName: \"kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:20.967785 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:20.967642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.068287 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.068250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6k2z\" (UniqueName: \"kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.068525 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.068344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.070860 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.070837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.076760 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.076737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6k2z\" (UniqueName: \"kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.250513 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.250426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:21.527416 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:21.527375    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:47:21.528530 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:47:21.528495    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60e4094_2569_46bd_8ef1_4da69f79c69d.slice/crio-380a2d4b9f93986d7eb431f8dad95f133cae987f2900e837feaa6cd8484b4dab WatchSource:0}: Error finding container 380a2d4b9f93986d7eb431f8dad95f133cae987f2900e837feaa6cd8484b4dab: Status 404 returned error can't find the container with id 380a2d4b9f93986d7eb431f8dad95f133cae987f2900e837feaa6cd8484b4dab
Sep 16 16:47:22.243397 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:22.243357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" event={"ID":"d60e4094-2569-46bd-8ef1-4da69f79c69d","Type":"ContainerStarted","Data":"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"}
Sep 16 16:47:22.243869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:22.243421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" event={"ID":"d60e4094-2569-46bd-8ef1-4da69f79c69d","Type":"ContainerStarted","Data":"380a2d4b9f93986d7eb431f8dad95f133cae987f2900e837feaa6cd8484b4dab"}
Sep 16 16:47:22.243869 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:22.243571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:47:22.262011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:22.261962    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podStartSLOduration=2.26194818 podStartE2EDuration="2.26194818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:47:22.260527562 +0000 UTC m=+3439.487985570" watchObservedRunningTime="2026-09-16 16:47:22.26194818 +0000 UTC m=+3439.489406186"
Sep 16 16:47:30.070729 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:30.070688    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:47:35.323689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.323653    2581 generic.go:356] "Generic (PLEG): container finished" podID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerID="d3bf7e375c3bc4ebbd3099bb9d42de58903c16eacc99da6f3bfa50961f0fd67b" exitCode=0
Sep 16 16:47:35.324057 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.323698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" event={"ID":"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17","Type":"ContainerDied","Data":"d3bf7e375c3bc4ebbd3099bb9d42de58903c16eacc99da6f3bfa50961f0fd67b"}
Sep 16 16:47:35.341084 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.341057    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:47:35.403123 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.403091    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs\") pod \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") "
Sep 16 16:47:35.403346 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.403199    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgjp2\" (UniqueName: \"kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2\") pod \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\" (UID: \"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17\") "
Sep 16 16:47:35.405476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.405452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:47:35.405476 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.405461    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2" (OuterVolumeSpecName: "kube-api-access-hgjp2") pod "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" (UID: "dc00d9e9-c81d-4af2-89bb-8ee43edc4e17"). InnerVolumeSpecName "kube-api-access-hgjp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:35.504574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.504529    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:47:35.504574 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:35.504560    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgjp2\" (UniqueName: \"kubernetes.io/projected/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17-kube-api-access-hgjp2\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:47:36.330174 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:36.330121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"
Sep 16 16:47:36.330600 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:36.330174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s" event={"ID":"dc00d9e9-c81d-4af2-89bb-8ee43edc4e17","Type":"ContainerDied","Data":"181cce837f569a1538f7de0840d34e377803a01ae34b9286d719ee5f1c58b5a0"}
Sep 16 16:47:36.330600 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:36.330241    2581 scope.go:117] "RemoveContainer" containerID="d3bf7e375c3bc4ebbd3099bb9d42de58903c16eacc99da6f3bfa50961f0fd67b"
Sep 16 16:47:36.351731 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:36.351698    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:47:36.355495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:36.355467    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-86dbc9cdb7-sn69s"]
Sep 16 16:47:37.310775 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:37.310737    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" path="/var/lib/kubelet/pods/dc00d9e9-c81d-4af2-89bb-8ee43edc4e17/volumes"
Sep 16 16:47:53.251923 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:47:53.251885    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:49:12.738188 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:12.738132    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:49:13.251086 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:13.251049    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:49:23.250883 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:23.250837    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:49:27.761749 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:27.761702    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" containerID="cri-o://bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00" gracePeriod=45
Sep 16 16:49:33.251148 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:33.251103    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:49:33.251559 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:33.251277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:49:41.941862 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.941826    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:49:41.942258 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.942238    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main"
Sep 16 16:49:41.942258 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.942252    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main"
Sep 16 16:49:41.942334 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.942315    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc00d9e9-c81d-4af2-89bb-8ee43edc4e17" containerName="main"
Sep 16 16:49:41.945385 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.945362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:41.948068 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.948038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-k4z2v\""
Sep 16 16:49:41.954075 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:41.954043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:49:42.013907 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.013882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l494r\" (UniqueName: \"kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.014073 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.013925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.115252 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.115220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l494r\" (UniqueName: \"kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.115429 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.115261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.117793 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.117764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.123732 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.123708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l494r\" (UniqueName: \"kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.257208 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.257099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:42.532741 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:42.532715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:49:42.534496 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:49:42.534466    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf634b0a2_a97e_4632_869b_551837d74330.slice/crio-21443e8fcf76452b6f37401f8979822c2ec5f44c0d4059b8b33b8110cd1a0797 WatchSource:0}: Error finding container 21443e8fcf76452b6f37401f8979822c2ec5f44c0d4059b8b33b8110cd1a0797: Status 404 returned error can't find the container with id 21443e8fcf76452b6f37401f8979822c2ec5f44c0d4059b8b33b8110cd1a0797
Sep 16 16:49:43.025550 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:43.025512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" event={"ID":"f634b0a2-a97e-4632-869b-551837d74330","Type":"ContainerStarted","Data":"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"}
Sep 16 16:49:43.026058 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:43.025571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" event={"ID":"f634b0a2-a97e-4632-869b-551837d74330","Type":"ContainerStarted","Data":"21443e8fcf76452b6f37401f8979822c2ec5f44c0d4059b8b33b8110cd1a0797"}
Sep 16 16:49:43.026058 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:43.025641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:49:43.044613 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:43.044563    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podStartSLOduration=2.044548962 podStartE2EDuration="2.044548962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:49:43.042488799 +0000 UTC m=+3580.269946806" watchObservedRunningTime="2026-09-16 16:49:43.044548962 +0000 UTC m=+3580.272006968"
Sep 16 16:49:43.251215 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:43.251173    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:49:53.251593 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:53.251554    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:49:58.008811 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.008785    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:49:58.056582 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.056499    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6k2z\" (UniqueName: \"kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z\") pod \"d60e4094-2569-46bd-8ef1-4da69f79c69d\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") "
Sep 16 16:49:58.056582 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.056563    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs\") pod \"d60e4094-2569-46bd-8ef1-4da69f79c69d\" (UID: \"d60e4094-2569-46bd-8ef1-4da69f79c69d\") "
Sep 16 16:49:58.058766 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.058740    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z" (OuterVolumeSpecName: "kube-api-access-d6k2z") pod "d60e4094-2569-46bd-8ef1-4da69f79c69d" (UID: "d60e4094-2569-46bd-8ef1-4da69f79c69d"). InnerVolumeSpecName "kube-api-access-d6k2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:49:58.058889 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.058779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d60e4094-2569-46bd-8ef1-4da69f79c69d" (UID: "d60e4094-2569-46bd-8ef1-4da69f79c69d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:49:58.114118 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.114086    2581 generic.go:356] "Generic (PLEG): container finished" podID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerID="bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00" exitCode=0
Sep 16 16:49:58.114290 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.114129    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"
Sep 16 16:49:58.114290 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.114135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" event={"ID":"d60e4094-2569-46bd-8ef1-4da69f79c69d","Type":"ContainerDied","Data":"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"}
Sep 16 16:49:58.114384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.114308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6" event={"ID":"d60e4094-2569-46bd-8ef1-4da69f79c69d","Type":"ContainerDied","Data":"380a2d4b9f93986d7eb431f8dad95f133cae987f2900e837feaa6cd8484b4dab"}
Sep 16 16:49:58.114384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.114348    2581 scope.go:117] "RemoveContainer" containerID="bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"
Sep 16 16:49:58.124149 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.124130    2581 scope.go:117] "RemoveContainer" containerID="bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"
Sep 16 16:49:58.124428 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:49:58.124407    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00\": container with ID starting with bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00 not found: ID does not exist" containerID="bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"
Sep 16 16:49:58.124478 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.124439    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00"} err="failed to get container status \"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00\": rpc error: code = NotFound desc = could not find container \"bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00\": container with ID starting with bf1e7ed8caedf3133317e5ff71861bab470a444f95755f505980ed089ac1be00 not found: ID does not exist"
Sep 16 16:49:58.137560 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.137533    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:49:58.143617 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.143592    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-ll8q6"]
Sep 16 16:49:58.157384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.157360    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6k2z\" (UniqueName: \"kubernetes.io/projected/d60e4094-2569-46bd-8ef1-4da69f79c69d-kube-api-access-d6k2z\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:49:58.157384 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:58.157381    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60e4094-2569-46bd-8ef1-4da69f79c69d-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:49:59.310065 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:49:59.310033    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" path="/var/lib/kubelet/pods/d60e4094-2569-46bd-8ef1-4da69f79c69d/volumes"
Sep 16 16:50:03.638570 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:50:03.638541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:50:03.644397 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:50:03.644363    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:50:03.646660 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:50:03.646639    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:50:03.652374 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:50:03.652356    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:50:14.034840 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:50:14.034808    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:51:04.445474 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:04.445434    2581 scope.go:117] "RemoveContainer" containerID="91c7101e84ad7108fd13218962c0515c5d1c920ced47da098a40753d7e20487f"
Sep 16 16:51:33.808579 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:33.808498    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:51:34.034756 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:34.034718    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:51:44.034855 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:44.034816    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:51:48.828368 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:48.828326    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" containerID="cri-o://bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2" gracePeriod=45
Sep 16 16:51:54.034144 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:54.034102    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:51:54.034583 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:51:54.034229    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:52:04.034414 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:04.034370    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:52:14.033862 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:14.033823    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 16:52:19.085743 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.085681    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:52:19.279492 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.279439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs\") pod \"f634b0a2-a97e-4632-869b-551837d74330\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") "
Sep 16 16:52:19.279689 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.279547    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l494r\" (UniqueName: \"kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r\") pod \"f634b0a2-a97e-4632-869b-551837d74330\" (UID: \"f634b0a2-a97e-4632-869b-551837d74330\") "
Sep 16 16:52:19.281576 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.281545    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f634b0a2-a97e-4632-869b-551837d74330" (UID: "f634b0a2-a97e-4632-869b-551837d74330"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:52:19.281725 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.281636    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r" (OuterVolumeSpecName: "kube-api-access-l494r") pod "f634b0a2-a97e-4632-869b-551837d74330" (UID: "f634b0a2-a97e-4632-869b-551837d74330"). InnerVolumeSpecName "kube-api-access-l494r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:19.380544 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.380518    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l494r\" (UniqueName: \"kubernetes.io/projected/f634b0a2-a97e-4632-869b-551837d74330-kube-api-access-l494r\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:52:19.380684 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.380549    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f634b0a2-a97e-4632-869b-551837d74330-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:52:19.868715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.868620    2581 generic.go:356] "Generic (PLEG): container finished" podID="f634b0a2-a97e-4632-869b-551837d74330" containerID="bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2" exitCode=0
Sep 16 16:52:19.868715 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.868664    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"
Sep 16 16:52:19.869032 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.868710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" event={"ID":"f634b0a2-a97e-4632-869b-551837d74330","Type":"ContainerDied","Data":"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"}
Sep 16 16:52:19.869032 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.868759    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z" event={"ID":"f634b0a2-a97e-4632-869b-551837d74330","Type":"ContainerDied","Data":"21443e8fcf76452b6f37401f8979822c2ec5f44c0d4059b8b33b8110cd1a0797"}
Sep 16 16:52:19.869032 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.868782    2581 scope.go:117] "RemoveContainer" containerID="bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"
Sep 16 16:52:19.877504 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.877487    2581 scope.go:117] "RemoveContainer" containerID="bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"
Sep 16 16:52:19.877751 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:52:19.877731    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2\": container with ID starting with bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2 not found: ID does not exist" containerID="bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"
Sep 16 16:52:19.877809 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.877758    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2"} err="failed to get container status \"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2\": rpc error: code = NotFound desc = could not find container \"bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2\": container with ID starting with bbd2254a67af0394f1507fc7d756d896fcfe1271a6ed717258ae562828074dc2 not found: ID does not exist"
Sep 16 16:52:19.891859 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.891789    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:52:19.892554 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:19.892534    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-mp72z"]
Sep 16 16:52:21.311110 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:52:21.311070    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f634b0a2-a97e-4632-869b-551837d74330" path="/var/lib/kubelet/pods/f634b0a2-a97e-4632-869b-551837d74330/volumes"
Sep 16 16:53:41.725627 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.725592    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.725964    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main"
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.725977    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main"
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.725985    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main"
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.725991    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main"
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.726052    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f634b0a2-a97e-4632-869b-551837d74330" containerName="main"
Sep 16 16:53:41.726229 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.726062    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d60e4094-2569-46bd-8ef1-4da69f79c69d" containerName="main"
Sep 16 16:53:41.729093 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.729072    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.732443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.732046    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Sep 16 16:53:41.732443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.732118    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Sep 16 16:53:41.732443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.732398    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Sep 16 16:53:41.732443 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.732441    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-gn95v\""
Sep 16 16:53:41.746226 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.746199    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:53:41.759770 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.759878 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.759924 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.759961 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.759961 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.760034 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.759994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.760072 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.760036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6s87\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.760072 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.760053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.760140 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.760080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.860996 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.860959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6s87\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861223 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861577 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861700 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861597    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861700 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861804 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.861941 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.862084 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.861981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.863794 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.863771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.863921 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.863903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.872285 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.872263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6s87\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:41.872367 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:41.872294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token\") pod \"router-gateway-1-openshift-default-56bd58cf4d-wt7fj\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:42.041751 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:42.041656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:42.323470 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:42.323441    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:53:42.324754 ip-10-0-137-111 kubenswrapper[2581]: W0916 16:53:42.324726    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73abd1b4_4998_4aaa_a429_f65daeefc93a.slice/crio-559de1e73f86c785ae7737338f0a23bea1ac7d2979a28959c132def0521e6217 WatchSource:0}: Error finding container 559de1e73f86c785ae7737338f0a23bea1ac7d2979a28959c132def0521e6217: Status 404 returned error can't find the container with id 559de1e73f86c785ae7737338f0a23bea1ac7d2979a28959c132def0521e6217
Sep 16 16:53:42.326855 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:42.326824    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:42.326957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:42.326884    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:42.326957 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:42.326916    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:43.311548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:43.311505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerStarted","Data":"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"}
Sep 16 16:53:43.311548 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:43.311553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerStarted","Data":"559de1e73f86c785ae7737338f0a23bea1ac7d2979a28959c132def0521e6217"}
Sep 16 16:53:43.335900 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:43.335380    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podStartSLOduration=2.335360567 podStartE2EDuration="2.335360567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:43.333552648 +0000 UTC m=+3820.561010655" watchObservedRunningTime="2026-09-16 16:53:43.335360567 +0000 UTC m=+3820.562818577"
Sep 16 16:53:44.042240 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:44.042205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:44.047806 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:44.047769    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:44.316923 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:44.316837    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:44.317838 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:44.317814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:53.372906 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.372804    2581 generic.go:356] "Generic (PLEG): container finished" podID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerID="619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c" exitCode=0
Sep 16 16:53:53.372906 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.372877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerDied","Data":"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"}
Sep 16 16:53:53.373371 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.373242    2581 scope.go:117] "RemoveContainer" containerID="619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"
Sep 16 16:53:53.374147 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.374116    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:53.374274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.374213    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:53.374274 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:53.374264    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 16:53:54.380631 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:54.380594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerStarted","Data":"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"}
Sep 16 16:53:55.042373 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:55.042334    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:53:55.043684 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:55.043645    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 16 16:53:56.042329 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:56.042287    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 16 16:53:57.042771 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:57.042726    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 16 16:53:58.042498 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:58.042447    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 16 16:53:59.042228 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:59.042182    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 16 16:53:59.317601 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:53:59.317525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:54:00.046546 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:54:00.046504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:54:00.047660 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:54:00.047635    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:55:03.674906 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:55:03.674870    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:55:03.680696 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:55:03.680673    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:55:03.684581 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:55:03.684559    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 16:55:03.696653 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:55:03.696631    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 16:56:58.641187 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:56:58.641133    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:56:58.641737 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:56:58.641401    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" containerID="cri-o://d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64" gracePeriod=30
Sep 16 16:57:00.049397 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:00.049349    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:57:03.894973 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.894945    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:57:03.947849 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.947819    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.947861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.947888    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948011 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.947914    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948187 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948187 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948080    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948187 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6s87\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:03.948349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:03.948349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data" (OuterVolumeSpecName: "istio-data") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:03.948349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948284    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948349 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948313    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token\") pod \"73abd1b4-4998-4aaa-a429-f65daeefc93a\" (UID: \"73abd1b4-4998-4aaa-a429-f65daeefc93a\") "
Sep 16 16:57:03.948614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:03.948614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948567    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:03.948614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948588    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-data\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:03.948614 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948602    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-workload-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:03.948827 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948617    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-credential-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:03.948913 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.948886    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:03.950866 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.950833    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:57:03.953647 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.953614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 16:57:03.953746 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.953650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87" (OuterVolumeSpecName: "kube-api-access-j6s87") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "kube-api-access-j6s87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:03.953849 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:03.953828    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token" (OuterVolumeSpecName: "istio-token") pod "73abd1b4-4998-4aaa-a429-f65daeefc93a" (UID: "73abd1b4-4998-4aaa-a429-f65daeefc93a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:04.050037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.049999    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-podinfo\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:04.050037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.050030    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-token\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:04.050037 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.050042    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73abd1b4-4998-4aaa-a429-f65daeefc93a-istiod-ca-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:04.050311 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.050050    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73abd1b4-4998-4aaa-a429-f65daeefc93a-istio-envoy\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:04.050311 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.050059    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6s87\" (UniqueName: \"kubernetes.io/projected/73abd1b4-4998-4aaa-a429-f65daeefc93a-kube-api-access-j6s87\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 16:57:04.407540 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.407500    2581 generic.go:356] "Generic (PLEG): container finished" podID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerID="d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64" exitCode=0
Sep 16 16:57:04.407720 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.407561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerDied","Data":"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"}
Sep 16 16:57:04.407720 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.407604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj" event={"ID":"73abd1b4-4998-4aaa-a429-f65daeefc93a","Type":"ContainerDied","Data":"559de1e73f86c785ae7737338f0a23bea1ac7d2979a28959c132def0521e6217"}
Sep 16 16:57:04.407720 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.407627    2581 scope.go:117] "RemoveContainer" containerID="d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"
Sep 16 16:57:04.407720 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.407561    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"
Sep 16 16:57:04.420893 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.420869    2581 scope.go:117] "RemoveContainer" containerID="619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"
Sep 16 16:57:04.429495 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.429470    2581 scope.go:117] "RemoveContainer" containerID="d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"
Sep 16 16:57:04.429771 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:57:04.429748    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64\": container with ID starting with d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64 not found: ID does not exist" containerID="d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"
Sep 16 16:57:04.429813 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.429782    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64"} err="failed to get container status \"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64\": rpc error: code = NotFound desc = could not find container \"d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64\": container with ID starting with d2b437a9b1fdd219311b0a471aea8ff636572fdd7a66023e16e14d62e0981d64 not found: ID does not exist"
Sep 16 16:57:04.429813 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.429801    2581 scope.go:117] "RemoveContainer" containerID="619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"
Sep 16 16:57:04.430051 ip-10-0-137-111 kubenswrapper[2581]: E0916 16:57:04.430031    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c\": container with ID starting with 619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c not found: ID does not exist" containerID="619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"
Sep 16 16:57:04.430098 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.430060    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c"} err="failed to get container status \"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c\": rpc error: code = NotFound desc = could not find container \"619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c\": container with ID starting with 619d93d2c3bb36772ddac766a935a7a342c0da71daf95f461c1e65063a01687c not found: ID does not exist"
Sep 16 16:57:04.436737 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.436714    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:57:04.441012 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:04.440984    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56bd58cf4d-wt7fj"]
Sep 16 16:57:05.310669 ip-10-0-137-111 kubenswrapper[2581]: I0916 16:57:05.310629    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" path="/var/lib/kubelet/pods/73abd1b4-4998-4aaa-a429-f65daeefc93a/volumes"
Sep 16 17:00:03.710325 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:00:03.710288    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:00:03.716483 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:00:03.716457    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:00:03.731094 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:00:03.731069    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:00:03.736838 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:00:03.736818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:01:37.170090 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.170049    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:01:37.170727 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.170702    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:01:37.170778 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.170732    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:01:37.170886 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.170873    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:01:37.170921 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.170892    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:01:37.174317 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.174293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.178281 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.178231    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-x9ct8\""
Sep 16 17:01:37.178412 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.178285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-67t67\""
Sep 16 17:01:37.178412 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.178240    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Sep 16 17:01:37.178412 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.178388    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Sep 16 17:01:37.178606 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.178508    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Sep 16 17:01:37.192278 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.188237    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:01:37.221248 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.221211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.221430 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.221298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrnhq\" (UniqueName: \"kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.321852 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.321814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrnhq\" (UniqueName: \"kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.322027 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.321873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.324567 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.324531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.330334 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.330310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrnhq\" (UniqueName: \"kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.490722 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.490620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.768275 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.768028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:01:37.770990 ip-10-0-137-111 kubenswrapper[2581]: W0916 17:01:37.770960    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06003824_8730_4651_a261_cffdbbe17bb6.slice/crio-9751ce99c2f937345aa838cd70714dc17abd805fb49797b4c9ce68bf9e643495 WatchSource:0}: Error finding container 9751ce99c2f937345aa838cd70714dc17abd805fb49797b4c9ce68bf9e643495: Status 404 returned error can't find the container with id 9751ce99c2f937345aa838cd70714dc17abd805fb49797b4c9ce68bf9e643495
Sep 16 17:01:37.860210 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.860174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" event={"ID":"06003824-8730-4651-a261-cffdbbe17bb6","Type":"ContainerStarted","Data":"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"}
Sep 16 17:01:37.860339 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.860239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" event={"ID":"06003824-8730-4651-a261-cffdbbe17bb6","Type":"ContainerStarted","Data":"9751ce99c2f937345aa838cd70714dc17abd805fb49797b4c9ce68bf9e643495"}
Sep 16 17:01:37.860390 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.860328    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:01:37.878068 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:01:37.878015    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podStartSLOduration=0.877998733 podStartE2EDuration="877.998733ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:01:37.876534478 +0000 UTC m=+4295.103992511" watchObservedRunningTime="2026-09-16 17:01:37.877998733 +0000 UTC m=+4295.105456739"
Sep 16 17:02:08.867564 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:02:08.867484    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:03:46.232257 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:03:46.232137    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:03:48.867648 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:03:48.867609    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:03:58.866926 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:03:58.866880    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:04:01.252470 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:01.252427    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" containerID="cri-o://b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484" gracePeriod=45
Sep 16 17:04:08.867339 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:08.867291    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:04:08.867801 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:08.867398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:04:18.867646 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:18.867601    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:04:28.866971 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:28.866925    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:04:31.496233 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.496207    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:04:31.613535 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.613430    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs\") pod \"06003824-8730-4651-a261-cffdbbe17bb6\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") "
Sep 16 17:04:31.613535 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.613485    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrnhq\" (UniqueName: \"kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq\") pod \"06003824-8730-4651-a261-cffdbbe17bb6\" (UID: \"06003824-8730-4651-a261-cffdbbe17bb6\") "
Sep 16 17:04:31.615775 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.615741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06003824-8730-4651-a261-cffdbbe17bb6" (UID: "06003824-8730-4651-a261-cffdbbe17bb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:04:31.615891 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.615782    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq" (OuterVolumeSpecName: "kube-api-access-mrnhq") pod "06003824-8730-4651-a261-cffdbbe17bb6" (UID: "06003824-8730-4651-a261-cffdbbe17bb6"). InnerVolumeSpecName "kube-api-access-mrnhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:31.714278 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.714244    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06003824-8730-4651-a261-cffdbbe17bb6-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:04:31.714278 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.714274    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrnhq\" (UniqueName: \"kubernetes.io/projected/06003824-8730-4651-a261-cffdbbe17bb6-kube-api-access-mrnhq\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:04:31.765089 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.765050    2581 generic.go:356] "Generic (PLEG): container finished" podID="06003824-8730-4651-a261-cffdbbe17bb6" containerID="b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484" exitCode=0
Sep 16 17:04:31.765314 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.765098    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"
Sep 16 17:04:31.765314 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.765120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" event={"ID":"06003824-8730-4651-a261-cffdbbe17bb6","Type":"ContainerDied","Data":"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"}
Sep 16 17:04:31.765314 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.765206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8" event={"ID":"06003824-8730-4651-a261-cffdbbe17bb6","Type":"ContainerDied","Data":"9751ce99c2f937345aa838cd70714dc17abd805fb49797b4c9ce68bf9e643495"}
Sep 16 17:04:31.765314 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.765237    2581 scope.go:117] "RemoveContainer" containerID="b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"
Sep 16 17:04:31.774369 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.774345    2581 scope.go:117] "RemoveContainer" containerID="b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"
Sep 16 17:04:31.774647 ip-10-0-137-111 kubenswrapper[2581]: E0916 17:04:31.774626    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484\": container with ID starting with b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484 not found: ID does not exist" containerID="b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"
Sep 16 17:04:31.774693 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.774656    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484"} err="failed to get container status \"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484\": rpc error: code = NotFound desc = could not find container \"b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484\": container with ID starting with b3c1f84666e6f5a7be079ab379939a034286b21cc165f75a1406c6f8e9638484 not found: ID does not exist"
Sep 16 17:04:31.787044 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.787014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:04:31.795939 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:31.792580    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48zljr8"]
Sep 16 17:04:33.311293 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:33.311251    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06003824-8730-4651-a261-cffdbbe17bb6" path="/var/lib/kubelet/pods/06003824-8730-4651-a261-cffdbbe17bb6/volumes"
Sep 16 17:04:36.555592 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.555553    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:04:36.556114 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.556090    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:04:36.556216 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.556119    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="73abd1b4-4998-4aaa-a429-f65daeefc93a" containerName="istio-proxy"
Sep 16 17:04:36.556216 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.556131    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main"
Sep 16 17:04:36.556216 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.556139    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main"
Sep 16 17:04:36.556382 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.556253    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="06003824-8730-4651-a261-cffdbbe17bb6" containerName="main"
Sep 16 17:04:36.560949 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.560927    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.563960 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.563892    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Sep 16 17:04:36.563960 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.563935    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-gateway-2-openshift-default-dockercfg-pxc86\""
Sep 16 17:04:36.564136 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.564036    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-gw-ca-root-cert\""
Sep 16 17:04:36.564218 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.564173    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Sep 16 17:04:36.573494 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.573470    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:04:36.656953 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.656910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657300 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657454 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657454 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657579 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657579 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657579 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657579 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.657781 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.657601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6thl\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.758787 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.758749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.758787 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.758792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759066 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.758923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759066 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.758970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759066 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6thl\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759066 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759325 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759325 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759325 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759493 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759493 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759493 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759692 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.759692 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.759673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.761284 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.761261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.761686 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.761667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.767206 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.767179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.767355 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.767335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6thl\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl\") pod \"router-gateway-2-openshift-default-5858dff66d-vmkss\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:36.872218 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:36.872098    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:37.154984 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.154947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:04:37.162425 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.162373    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:37.162542 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.162469    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:37.162542 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.162511    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:37.803053 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.803013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerStarted","Data":"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"}
Sep 16 17:04:37.803437 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.803070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerStarted","Data":"fb845f184d701e219b66a0359621badec7346935529981efc57f45839d5b81fb"}
Sep 16 17:04:37.823900 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.823844    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podStartSLOduration=1.823828878 podStartE2EDuration="1.823828878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:04:37.822947044 +0000 UTC m=+4475.050405046" watchObservedRunningTime="2026-09-16 17:04:37.823828878 +0000 UTC m=+4475.051286884"
Sep 16 17:04:37.872687 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:37.872651    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:38.878318 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:38.878282    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:39.814501 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:39.814460    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:39.815498 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:39.815468    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:46.549755 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.549717    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:04:46.553222 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.553199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.557439 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.557410    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-qr8qg\""
Sep 16 17:04:46.557439 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.557428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Sep 16 17:04:46.557641 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.557517    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-2vbbh\""
Sep 16 17:04:46.561882 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.561846    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:04:46.647115 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.647058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbfn4\" (UniqueName: \"kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.647354 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.647134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.748353 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.748311    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.748623 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.748440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbfn4\" (UniqueName: \"kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.751242 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.751208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.756374 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.756344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbfn4\" (UniqueName: \"kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:46.866392 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:46.866299    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:47.143544 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:47.143513    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:04:47.145942 ip-10-0-137-111 kubenswrapper[2581]: W0916 17:04:47.145910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25da006f_0b2e_4016_8483_e9931e7487cd.slice/crio-2a7ced61eddb28ee909df7df2f0c738fd9160872cc54a8c48cded6e9fb730db5 WatchSource:0}: Error finding container 2a7ced61eddb28ee909df7df2f0c738fd9160872cc54a8c48cded6e9fb730db5: Status 404 returned error can't find the container with id 2a7ced61eddb28ee909df7df2f0c738fd9160872cc54a8c48cded6e9fb730db5
Sep 16 17:04:47.860541 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:47.860487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" event={"ID":"25da006f-0b2e-4016-8483-e9931e7487cd","Type":"ContainerStarted","Data":"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"}
Sep 16 17:04:47.861002 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:47.860568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:04:47.861002 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:47.860593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" event={"ID":"25da006f-0b2e-4016-8483-e9931e7487cd","Type":"ContainerStarted","Data":"2a7ced61eddb28ee909df7df2f0c738fd9160872cc54a8c48cded6e9fb730db5"}
Sep 16 17:04:47.877736 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:47.877680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podStartSLOduration=1.877663794 podStartE2EDuration="1.877663794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:04:47.876519436 +0000 UTC m=+4485.103977446" watchObservedRunningTime="2026-09-16 17:04:47.877663794 +0000 UTC m=+4485.105121824"
Sep 16 17:04:48.867998 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.867902    2581 generic.go:356] "Generic (PLEG): container finished" podID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerID="2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f" exitCode=0
Sep 16 17:04:48.867998 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.867979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerDied","Data":"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"}
Sep 16 17:04:48.868623 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.868466    2581 scope.go:117] "RemoveContainer" containerID="2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"
Sep 16 17:04:48.869395 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.869364    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:48.869493 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.869470    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:48.869556 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:48.869504    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 16 17:04:49.876239 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:49.876200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerStarted","Data":"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"}
Sep 16 17:04:50.873047 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:50.872998    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:50.874335 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:50.874297    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:04:51.873354 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:51.873305    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:04:52.873403 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:52.873356    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:04:53.873528 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:53.873471    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:04:54.815210 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:54.815171    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:54.873614 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:54.873562    2581 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:04:55.876993 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:55.876911    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:04:55.877993 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:04:55.877969    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:05:03.753514 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:05:03.753474    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:05:03.759301 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:05:03.759276    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:05:03.769259 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:05:03.769235    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:05:03.775124 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:05:03.775102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:05:18.870002 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:05:18.869964    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:06:55.754546 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:06:55.754504    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:06:58.870258 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:06:58.870217    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:07:08.870487 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:08.870448    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:07:10.774548 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:10.774503    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" containerID="cri-o://047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738" gracePeriod=45
Sep 16 17:07:18.870535 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:18.870490    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:07:18.871117 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:18.870639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:07:28.870714 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:28.870670    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:07:38.870067 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:38.870020    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:07:41.028243 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.028210    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:07:41.189120 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.189082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs\") pod \"25da006f-0b2e-4016-8483-e9931e7487cd\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") "
Sep 16 17:07:41.189329 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.189215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbfn4\" (UniqueName: \"kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4\") pod \"25da006f-0b2e-4016-8483-e9931e7487cd\" (UID: \"25da006f-0b2e-4016-8483-e9931e7487cd\") "
Sep 16 17:07:41.191420 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.191389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4" (OuterVolumeSpecName: "kube-api-access-nbfn4") pod "25da006f-0b2e-4016-8483-e9931e7487cd" (UID: "25da006f-0b2e-4016-8483-e9931e7487cd"). InnerVolumeSpecName "kube-api-access-nbfn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:41.191550 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.191495    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25da006f-0b2e-4016-8483-e9931e7487cd" (UID: "25da006f-0b2e-4016-8483-e9931e7487cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:07:41.290108 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.290066    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25da006f-0b2e-4016-8483-e9931e7487cd-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:41.290108 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.290104    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbfn4\" (UniqueName: \"kubernetes.io/projected/25da006f-0b2e-4016-8483-e9931e7487cd-kube-api-access-nbfn4\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:41.798397 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.798357    2581 generic.go:356] "Generic (PLEG): container finished" podID="25da006f-0b2e-4016-8483-e9931e7487cd" containerID="047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738" exitCode=0
Sep 16 17:07:41.798563 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.798404    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"
Sep 16 17:07:41.798563 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.798419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" event={"ID":"25da006f-0b2e-4016-8483-e9931e7487cd","Type":"ContainerDied","Data":"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"}
Sep 16 17:07:41.798563 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.798462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf" event={"ID":"25da006f-0b2e-4016-8483-e9931e7487cd","Type":"ContainerDied","Data":"2a7ced61eddb28ee909df7df2f0c738fd9160872cc54a8c48cded6e9fb730db5"}
Sep 16 17:07:41.798563 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.798486    2581 scope.go:117] "RemoveContainer" containerID="047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"
Sep 16 17:07:41.807851 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.807831    2581 scope.go:117] "RemoveContainer" containerID="047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"
Sep 16 17:07:41.808144 ip-10-0-137-111 kubenswrapper[2581]: E0916 17:07:41.808119    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738\": container with ID starting with 047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738 not found: ID does not exist" containerID="047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"
Sep 16 17:07:41.808249 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.808179    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738"} err="failed to get container status \"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738\": rpc error: code = NotFound desc = could not find container \"047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738\": container with ID starting with 047c072fa866a6d477aa8eeb329752f51dfffa3cd1a5bc1eed0f6b2e61c3e738 not found: ID does not exist"
Sep 16 17:07:41.816780 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.816748    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:07:41.818819 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:41.818796    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45cmhrf"]
Sep 16 17:07:43.310847 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:43.310815    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" path="/var/lib/kubelet/pods/25da006f-0b2e-4016-8483-e9931e7487cd/volumes"
Sep 16 17:07:50.860341 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:50.860299    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:07:50.860887 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:50.860532    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" containerID="cri-o://dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449" gracePeriod=30
Sep 16 17:07:55.878805 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:55.878769    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Sep 16 17:07:56.118653 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.118581    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:07:56.216322 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216483 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216333    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216483 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216483 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216483 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216528    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6thl\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216579    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token\") pod \"99e5ff07-0a51-4231-bee4-5e88f0493efc\" (UID: \"99e5ff07-0a51-4231-bee4-5e88f0493efc\") "
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216584    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:07:56.216669 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216630    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data" (OuterVolumeSpecName: "istio-data") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:07:56.216983 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216880    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.216983 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216902    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-data\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.216983 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:56.217145 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.216978    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:07:56.217145 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.217098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:07:56.218634 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.218604    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl" (OuterVolumeSpecName: "kube-api-access-l6thl") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "kube-api-access-l6thl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:56.218634 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.218628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 17:07:56.219049 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.219027    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:07:56.219117 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.219029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token" (OuterVolumeSpecName: "istio-token") pod "99e5ff07-0a51-4231-bee4-5e88f0493efc" (UID: "99e5ff07-0a51-4231-bee4-5e88f0493efc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:56.318275 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318240    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-envoy\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318275 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318270    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-workload-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318275 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318283    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99e5ff07-0a51-4231-bee4-5e88f0493efc-credential-socket\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318514 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318292    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99e5ff07-0a51-4231-bee4-5e88f0493efc-istiod-ca-cert\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318514 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318303    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6thl\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-kube-api-access-l6thl\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318514 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318312    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-token\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.318514 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.318320    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99e5ff07-0a51-4231-bee4-5e88f0493efc-istio-podinfo\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:07:56.883409 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.883369    2581 generic.go:356] "Generic (PLEG): container finished" podID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerID="dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449" exitCode=0
Sep 16 17:07:56.883856 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.883426    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"
Sep 16 17:07:56.883856 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.883433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerDied","Data":"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"}
Sep 16 17:07:56.883856 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.883488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss" event={"ID":"99e5ff07-0a51-4231-bee4-5e88f0493efc","Type":"ContainerDied","Data":"fb845f184d701e219b66a0359621badec7346935529981efc57f45839d5b81fb"}
Sep 16 17:07:56.883856 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.883517    2581 scope.go:117] "RemoveContainer" containerID="dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"
Sep 16 17:07:56.893062 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.893042    2581 scope.go:117] "RemoveContainer" containerID="2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"
Sep 16 17:07:56.903845 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.903060    2581 scope.go:117] "RemoveContainer" containerID="dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"
Sep 16 17:07:56.903845 ip-10-0-137-111 kubenswrapper[2581]: E0916 17:07:56.903577    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449\": container with ID starting with dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449 not found: ID does not exist" containerID="dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"
Sep 16 17:07:56.903845 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.903617    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449"} err="failed to get container status \"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449\": rpc error: code = NotFound desc = could not find container \"dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449\": container with ID starting with dcef20f01ef610431c121414a58410ff5a4a0d775fc562f4810242802ee38449 not found: ID does not exist"
Sep 16 17:07:56.903845 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.903639    2581 scope.go:117] "RemoveContainer" containerID="2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"
Sep 16 17:07:56.904194 ip-10-0-137-111 kubenswrapper[2581]: E0916 17:07:56.904148    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f\": container with ID starting with 2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f not found: ID does not exist" containerID="2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"
Sep 16 17:07:56.904243 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.904206    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f"} err="failed to get container status \"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f\": rpc error: code = NotFound desc = could not find container \"2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f\": container with ID starting with 2ccf69b9fbe2c1c2b8fb2ad49b39238497e3dc3a89714f323a2b8ccd2d72816f not found: ID does not exist"
Sep 16 17:07:56.944679 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.940053    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:07:56.950937 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:56.950906    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-vmkss"]
Sep 16 17:07:57.310205 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:07:57.310093    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" path="/var/lib/kubelet/pods/99e5ff07-0a51-4231-bee4-5e88f0493efc/volumes"
Sep 16 17:09:44.136403 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136362    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136712    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136724    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136736    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136743    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136827    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136838    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:09:44.137040 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.136845    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25da006f-0b2e-4016-8483-e9931e7487cd" containerName="main"
Sep 16 17:09:44.139867 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.139842    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.143999 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.143965    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-jbn9c\""
Sep 16 17:09:44.144139 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.144045    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Sep 16 17:09:44.144139 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.144062    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-epp-sa-dockercfg-95gt4\""
Sep 16 17:09:44.144139 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.144083    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Sep 16 17:09:44.144139 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.144084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Sep 16 17:09:44.149630 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.149607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:09:44.258279 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.258237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.258460 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.258300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9d9b\" (UniqueName: \"kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.359623 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.359584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.359819 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.359640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9d9b\" (UniqueName: \"kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.362264 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.362217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.367286 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.367257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9d9b\" (UniqueName: \"kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b\") pod \"rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.451002 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.450899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:44.730214 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:44.730130    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:09:44.733059 ip-10-0-137-111 kubenswrapper[2581]: W0916 17:09:44.733019    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d9ed4d_fd24_449c_b3d0_4fec1c8867d1.slice/crio-b6477217939429326d6fc6b0cab11e562847285837baae124473a4a3ff6ab631 WatchSource:0}: Error finding container b6477217939429326d6fc6b0cab11e562847285837baae124473a4a3ff6ab631: Status 404 returned error can't find the container with id b6477217939429326d6fc6b0cab11e562847285837baae124473a4a3ff6ab631
Sep 16 17:09:45.459025 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:45.458983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" event={"ID":"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1","Type":"ContainerStarted","Data":"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"}
Sep 16 17:09:45.459427 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:45.459039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" event={"ID":"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1","Type":"ContainerStarted","Data":"b6477217939429326d6fc6b0cab11e562847285837baae124473a4a3ff6ab631"}
Sep 16 17:09:45.459427 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:45.459084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:09:45.475397 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:09:45.475341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podStartSLOduration=1.475322776 podStartE2EDuration="1.475322776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:09:45.474794235 +0000 UTC m=+4782.702252246" watchObservedRunningTime="2026-09-16 17:09:45.475322776 +0000 UTC m=+4782.702780781"
Sep 16 17:10:03.794131 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:03.794096    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:10:03.800146 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:03.800119    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:10:03.807740 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:03.807719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:10:03.813240 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:03.813221    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lnfvq_55a9eb53-a0ca-46a4-bcad-37c5deb99e3b/ovn-acl-logging/0.log"
Sep 16 17:10:16.466503 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:16.466469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:10:26.913457 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:26.913412    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:10:36.466693 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:36.466648    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:10:41.934858 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:41.934812    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main" containerID="cri-o://9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006" gracePeriod=45
Sep 16 17:10:46.466800 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:46.466755    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:10:56.466596 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:56.466495    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:10:56.467012 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:10:56.466649    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:11:06.466980 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:06.466937    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 16 17:11:12.185234 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.185206    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:11:12.267283 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.267189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs\") pod \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") "
Sep 16 17:11:12.267283 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.267248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9d9b\" (UniqueName: \"kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b\") pod \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\" (UID: \"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1\") "
Sep 16 17:11:12.269448 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.269409    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" (UID: "31d9ed4d-fd24-449c-b3d0-4fec1c8867d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:11:12.269583 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.269465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b" (OuterVolumeSpecName: "kube-api-access-c9d9b") pod "31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" (UID: "31d9ed4d-fd24-449c-b3d0-4fec1c8867d1"). InnerVolumeSpecName "kube-api-access-c9d9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:11:12.368252 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.368213    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-tls-certs\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:11:12.368252 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.368252    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9d9b\" (UniqueName: \"kubernetes.io/projected/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1-kube-api-access-c9d9b\") on node \"ip-10-0-137-111.ec2.internal\" DevicePath \"\""
Sep 16 17:11:12.932690 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.932650    2581 generic.go:356] "Generic (PLEG): container finished" podID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerID="9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006" exitCode=0
Sep 16 17:11:12.932896 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.932702    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"
Sep 16 17:11:12.932896 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.932717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" event={"ID":"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1","Type":"ContainerDied","Data":"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"}
Sep 16 17:11:12.932896 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.932761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb" event={"ID":"31d9ed4d-fd24-449c-b3d0-4fec1c8867d1","Type":"ContainerDied","Data":"b6477217939429326d6fc6b0cab11e562847285837baae124473a4a3ff6ab631"}
Sep 16 17:11:12.932896 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.932784    2581 scope.go:117] "RemoveContainer" containerID="9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"
Sep 16 17:11:12.942453 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.942435    2581 scope.go:117] "RemoveContainer" containerID="9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"
Sep 16 17:11:12.942730 ip-10-0-137-111 kubenswrapper[2581]: E0916 17:11:12.942706    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006\": container with ID starting with 9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006 not found: ID does not exist" containerID="9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"
Sep 16 17:11:12.942774 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.942740    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006"} err="failed to get container status \"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006\": rpc error: code = NotFound desc = could not find container \"9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006\": container with ID starting with 9293be0eec8592409548a1a5948cdcb07f9d38b923370e501219316d3f721006 not found: ID does not exist"
Sep 16 17:11:12.954844 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.954814    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:11:12.960452 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:12.960429    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-74c7697f55-gp6jb"]
Sep 16 17:11:13.311599 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:13.311515    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" path="/var/lib/kubelet/pods/31d9ed4d-fd24-449c-b3d0-4fec1c8867d1/volumes"
Sep 16 17:11:39.246636 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:39.246598    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fddwk_02538c77-74c8-4198-816f-d3aca553961e/discovery/0.log"
Sep 16 17:11:40.011859 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:40.011823    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fddwk_02538c77-74c8-4198-816f-d3aca553961e/discovery/0.log"
Sep 16 17:11:41.086711 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:41.086676    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d9898bd99-ktrw6_a7003126-32d9-480a-b8ee-0de08daacbec/authorino/0.log"
Sep 16 17:11:41.099032 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:41.099009    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-q4fm2_832d05d3-8127-4771-a6bb-be10a99f6681/manager/0.log"
Sep 16 17:11:41.251792 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:41.251755    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-gbgm6_fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb/manager/0.log"
Sep 16 17:11:50.267082 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:50.267044    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hsntg_08ec670f-7702-4bae-a7c9-2e94c3b511b5/global-pull-secret-syncer/0.log"
Sep 16 17:11:50.371371 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:50.371339    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-krlk5_a760ae3f-642b-4210-b55b-6809037b13fa/konnectivity-agent/0.log"
Sep 16 17:11:50.473697 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:50.473666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-111.ec2.internal_6b7775f07a812f020e26327f5b0a385a/haproxy/0.log"
Sep 16 17:11:55.759622 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:55.759580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d9898bd99-ktrw6_a7003126-32d9-480a-b8ee-0de08daacbec/authorino/0.log"
Sep 16 17:11:55.795581 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:55.795547    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-q4fm2_832d05d3-8127-4771-a6bb-be10a99f6681/manager/0.log"
Sep 16 17:11:56.042062 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:56.041978    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6d4cb79c-gbgm6_fb4bea24-1cfe-4069-bb22-3bffe0e3d9bb/manager/0.log"
Sep 16 17:11:57.453388 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.453352    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nqbj5_8142c874-9e64-4134-8c29-68e5156c1e1a/kube-state-metrics/0.log"
Sep 16 17:11:57.478289 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.478262    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nqbj5_8142c874-9e64-4134-8c29-68e5156c1e1a/kube-rbac-proxy-main/0.log"
Sep 16 17:11:57.508489 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.508456    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-nqbj5_8142c874-9e64-4134-8c29-68e5156c1e1a/kube-rbac-proxy-self/0.log"
Sep 16 17:11:57.584812 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.584783    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-578pw_e865b252-175e-4808-8fa9-c81e392f6ce5/monitoring-plugin/0.log"
Sep 16 17:11:57.619018 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.618986    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vzf9_7b7cc6e7-d253-46c8-b6be-6de16054d773/node-exporter/0.log"
Sep 16 17:11:57.644543 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.644516    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vzf9_7b7cc6e7-d253-46c8-b6be-6de16054d773/kube-rbac-proxy/0.log"
Sep 16 17:11:57.667181 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:57.667082    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5vzf9_7b7cc6e7-d253-46c8-b6be-6de16054d773/init-textfile/0.log"
Sep 16 17:11:58.138420 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:58.138387    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-s99hx_8bdafc9f-0860-46eb-959e-251343192dc1/prometheus-operator/0.log"
Sep 16 17:11:58.161121 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:58.161087    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-s99hx_8bdafc9f-0860-46eb-959e-251343192dc1/kube-rbac-proxy/0.log"
Sep 16 17:11:58.186918 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:58.186887    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-fh77c_bf521d31-663b-41a2-8626-6bc53748c479/prometheus-operator-admission-webhook/0.log"
Sep 16 17:11:59.505319 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:11:59.505287    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-pbrk9_72cacd54-d295-4c94-b9bd-79cd8c9fd6b9/networking-console-plugin/0.log"
Sep 16 17:12:00.046697 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.046655    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"]
Sep 16 17:12:00.047120 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.047103    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main"
Sep 16 17:12:00.047199 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.047122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main"
Sep 16 17:12:00.047199 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.047138    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:12:00.047199 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.047143    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e5ff07-0a51-4231-bee4-5e88f0493efc" containerName="istio-proxy"
Sep 16 17:12:00.047308 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.047211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d9ed4d-fd24-449c-b3d0-4fec1c8867d1" containerName="main"
Sep 16 17:12:00.050242 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.050223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.051550 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.051525    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/2.log"
Sep 16 17:12:00.052703 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.052680    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgcmz\"/\"kube-root-ca.crt\""
Sep 16 17:12:00.052703 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.052686    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgcmz\"/\"openshift-service-ca.crt\""
Sep 16 17:12:00.052881 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.052756    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zgcmz\"/\"default-dockercfg-lmlsv\""
Sep 16 17:12:00.059760 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.059731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"]
Sep 16 17:12:00.060570 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.060549    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-64ghg_c46801f9-26da-47f0-aedb-97c459b68ef8/console-operator/3.log"
Sep 16 17:12:00.159081 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.159041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-lib-modules\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.159081 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.159085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-podres\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.159359 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.159138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-sys\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.159359 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.159249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-proc\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.159359 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.159297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sfr4\" (UniqueName: \"kubernetes.io/projected/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-kube-api-access-5sfr4\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260372 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-lib-modules\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260372 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-podres\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-sys\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-proc\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sfr4\" (UniqueName: \"kubernetes.io/projected/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-kube-api-access-5sfr4\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-sys\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-proc\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-podres\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.260639 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.260527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-lib-modules\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.268520 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.268490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sfr4\" (UniqueName: \"kubernetes.io/projected/7bfc6071-23ef-4d95-bd54-294bf7fe3ad5-kube-api-access-5sfr4\") pod \"perf-node-gather-daemonset-bb2cb\" (UID: \"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5\") " pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.362912 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.362813    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:00.565960 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.565926    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-757dd6b8c8-l2fhw_a4b0cc5b-faf2-4cac-ab1a-4886fe01dab7/console/0.log"
Sep 16 17:12:00.636710 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:00.636676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"]
Sep 16 17:12:00.640035 ip-10-0-137-111 kubenswrapper[2581]: W0916 17:12:00.640004    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7bfc6071_23ef_4d95_bd54_294bf7fe3ad5.slice/crio-44d8c145d27b7cf9ce23a4d6fba2cf86efaaf999796faf8440ccd7f3570558fd WatchSource:0}: Error finding container 44d8c145d27b7cf9ce23a4d6fba2cf86efaaf999796faf8440ccd7f3570558fd: Status 404 returned error can't find the container with id 44d8c145d27b7cf9ce23a4d6fba2cf86efaaf999796faf8440ccd7f3570558fd
Sep 16 17:12:01.165823 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:01.165794    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-c257x_70b27a36-718e-4e5c-bd85-a9a8844ae0ff/volume-data-source-validator/0.log"
Sep 16 17:12:01.189551 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:01.189511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb" event={"ID":"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5","Type":"ContainerStarted","Data":"707b2e34d23f9993cda0db10f29dbccb045b4e95758f2b030c09b0bb584e44db"}
Sep 16 17:12:01.189720 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:01.189570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb" event={"ID":"7bfc6071-23ef-4d95-bd54-294bf7fe3ad5","Type":"ContainerStarted","Data":"44d8c145d27b7cf9ce23a4d6fba2cf86efaaf999796faf8440ccd7f3570558fd"}
Sep 16 17:12:01.189720 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:01.189591    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:01.206032 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:01.205984    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb" podStartSLOduration=1.205969028 podStartE2EDuration="1.205969028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:12:01.20439392 +0000 UTC m=+4918.431851928" watchObservedRunningTime="2026-09-16 17:12:01.205969028 +0000 UTC m=+4918.433427034"
Sep 16 17:12:02.318586 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:02.318553    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdwrr_39025b22-64ea-4d4b-a568-69ccf39c1e3f/dns/0.log"
Sep 16 17:12:02.379060 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:02.379029    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdwrr_39025b22-64ea-4d4b-a568-69ccf39c1e3f/kube-rbac-proxy/0.log"
Sep 16 17:12:02.529643 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:02.529607    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rhqrq_b6a2a639-00a1-46a0-a895-bb7f6c89c275/dns-node-resolver/0.log"
Sep 16 17:12:03.058392 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:03.058353    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7dfd56794f-w8frz_b60d0094-bb1b-43a3-a346-fc66a95268ba/registry/0.log"
Sep 16 17:12:03.138207 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:03.138172    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v7nh6_1cb09db2-2a8a-48a2-8270-ac314de8cc96/node-ca/0.log"
Sep 16 17:12:04.217366 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:04.217332    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fddwk_02538c77-74c8-4198-816f-d3aca553961e/discovery/0.log"
Sep 16 17:12:04.850746 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:04.850710    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hbb4s_22113a5f-d398-4e16-b11e-b96c75d5e2e9/serve-healthcheck-canary/0.log"
Sep 16 17:12:05.428138 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:05.428103    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwdhs_3f17499d-742a-4812-90b1-bed1d98b1294/kube-rbac-proxy/0.log"
Sep 16 17:12:05.449970 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:05.449939    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwdhs_3f17499d-742a-4812-90b1-bed1d98b1294/exporter/0.log"
Sep 16 17:12:05.470654 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:05.470621    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwdhs_3f17499d-742a-4812-90b1-bed1d98b1294/extractor/0.log"
Sep 16 17:12:07.203982 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:07.203950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zgcmz/perf-node-gather-daemonset-bb2cb"
Sep 16 17:12:08.503650 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:08.503618    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-n9hpl_e3f2b8ea-adb3-44ec-b215-92fb259fc64b/openshift-lws-operator/0.log"
Sep 16 17:12:09.231599 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:09.231563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-74fc95bbfc-bphgb_a116d94b-fe7f-4fd1-b458-42e0167762ad/manager/0.log"
Sep 16 17:12:09.536107 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:09.536069    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fddwk_02538c77-74c8-4198-816f-d3aca553961e/discovery/0.log"
Sep 16 17:12:10.929501 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:10.929454    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fddwk_02538c77-74c8-4198-816f-d3aca553961e/discovery/0.log"
Sep 16 17:12:11.006802 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:11.006760    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-kvq8k_95edd79f-f65d-4a62-bcbf-bef82c5ea794/manager/0.log"
Sep 16 17:12:16.418107 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:16.418068    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-rcphw_fa9bf1a4-1949-4346-bdb4-88795e371dd2/kube-storage-version-migrator-operator/1.log"
Sep 16 17:12:16.418857 ip-10-0-137-111 kubenswrapper[2581]: I0916 17:12:16.418836    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-rcphw_fa9bf1a4-1949-4346-bdb4-88795e371dd2/kube-storage-version-migrator-operator/0.log"